PDA

View Full Version : GLSL determinant() returns an 'undefined' result



_NoName_
05-20-2017, 04:42 AM
I have a little function that should get the intersection point of a ray with an infinite plane.
The function looks like this :


vec3 raytracePlaneI(vec3 s, vec3 a, vec3 b, vec3 o, vec3 d)
{
float lU;
float lV;
float lT;
vec3 lN;

vec3 A = s;
vec3 B = a;
vec3 C = b;

vec3 R = o-A; //translation

mat3 matrix = mat3(B, C, -d);
mat3 matU = mat3(R, C, -d);
mat3 matV = mat3(B, R, -d);
mat3 matT = mat3(B, C, R);

float detM = determinant(matrix);

float detU = determinant(matU);
lU = detU/detM;


float detV = determinant(matV);
lV = detV/detM;


float detT = determinant(matT);
lT = detT/detM;

if(detU > 0)
{
resultColor = vec4(1, 0, 0, 1);
}
else if(detU < 0)
{
resultColor = vec4(0, 1, 0, 1);
}
else if(detU == 0)
{
resultColor = vec4(0, 0, 1, 1);
}
else
{
resultColor = vec4(1, 1, 1, 1);
}

return vec3(lU, lV, lT);
}

In the last part of the function I set the result color to an other color dependent of the value of detU.
I test for >, < and == 0 and it always takes the else part (white color).

Do anyone know whats the error ??

GClements
05-20-2017, 04:59 AM
I test for >, < and == 0 and it always takes the else part (white color).

That suggests that detU is NaN. That would imply that at least one element of the matrix is NaN.

Alternatively, something elsewhere in your code may be triggering undefined behaviour, resulting in resultColor not getting assigned.

_NoName_
05-21-2017, 04:27 AM
The problem was that did pow(-x, y) to calculate an argument for the function and this resulted in a NaN.
thank you.