intersection test, ray obb inverse model matrix
let's say I've got an object bounding box and a ray. And I want to test wether the ray
intersects the box or not. So I take the inverse matrix of the obb and multiply the
ray's position and ray's direction with it. Then I perform a ray aabb intersection. The obb
has been modeled in object space and so I can use this as an aabb for the intersection test.
So far that works. But I remember that normals should rotated by the inverse transpose
of the model matrix. And I think we can interpret the ray direction as a normal. Since the
ray position is a position I tried to multiply the ray position with the inverse and the ray direction
with the inverse transpose. But then there is no intersection. If I multiply both, the ray position
and the ray direction with the inverse or the inverse transpose then there is an intersection.
For the case of ray obb intersection in the way described above, do I need the inverse transpose ?
It only works if the ray position and ray direction gets multiplied with the same matrix (inverse or inverse transpose).
But it doesn't work if the ray position us multiplied with the inverse and the ray direction is multiplied with the inverse transpose.