# GLSL determinant() returns an 'undefined' result

• 05-20-2017, 04:42 AM
_NoName_
GLSL determinant() returns an 'undefined' result
I have a little function that should get the intersection point of a ray with an infinite plane.
The function looks like this :

Code :

```vec3 raytracePlaneI(vec3 s, vec3 a, vec3 b, vec3 o, vec3 d) { float lU; float lV; float lT; vec3 lN;   vec3 A = s; vec3 B = a; vec3 C = b;   vec3 R = o-A; //translation   mat3 matrix = mat3(B, C, -d); mat3 matU = mat3(R, C, -d); mat3 matV = mat3(B, R, -d); mat3 matT = mat3(B, C, R);   float detM = determinant(matrix);   float detU = determinant(matU); lU = detU/detM;     float detV = determinant(matV); lV = detV/detM;     float detT = determinant(matT); lT = detT/detM;   if(detU > 0) { resultColor = vec4(1, 0, 0, 1); } else if(detU < 0) { resultColor = vec4(0, 1, 0, 1); } else if(detU == 0) { resultColor = vec4(0, 0, 1, 1); } else { resultColor = vec4(1, 1, 1, 1); }   return vec3(lU, lV, lT); }```

In the last part of the function I set the result color to an other color dependent of the value of detU.
I test for >, < and == 0 and it always takes the else part (white color).

Do anyone know whats the error ??
• 05-20-2017, 04:59 AM
GClements
Quote:

Originally Posted by _NoName_
I test for >, < and == 0 and it always takes the else part (white color).

That suggests that detU is NaN. That would imply that at least one element of the matrix is NaN.

Alternatively, something elsewhere in your code may be triggering undefined behaviour, resulting in resultColor not getting assigned.
• 05-21-2017, 04:27 AM
_NoName_
The problem was that did pow(-x, y) to calculate an argument for the function and this resulted in a NaN.
thank you.