'undefined' pixels in float texture fragmentshader

Hi all,

I come to this forum bearing a big problem in a fragment shader
that I’m working on - I’ve cornered it into this little piece
of code that gives me pixels that are neither a value, nor #inf
nor #nan, ( are they discarded ? ). my main function looks like:

1.vec4 flow = texture2D( VelDenMap, uv );
2.float p = flow.z;

3.float ad = smoothstep( 0.0, advection, clamp(p, 0.0, 1.0) )
4.gl_FragData[0] = vec4( ad );

advection is 0.8, VelDenMap is a 32-bit float texture
which is initialised to 0.0 ( im sure of that )
now, when I render the resulting float32-texture with a shader
that displays each pixel as

p = ( p - lower ) / ( higher - lower )

where lower and higher are the upper-and lower bounds for the range of values i want to display, everything outside of the
range [0,1] will be shown as a noise-texture, the image
remains black, even setting lower = higher = -10.0, or
lower = higher = 10.0.

however, when I change line 4 into

4.gl_FragData[0] = vec4( p );

all is fine, proving that p is a legal value. the same goes
for

4.gl_FragData[0] = vec4( advection );

also checking against NAN with

float inf1 = 1.0 / 0.0;
float inf2 = 1.0 / 0.0;
float nAn = inf1 * inf2;
if( ad == nAn )…
if( ad == inf1 )…

doesn’t detect ‘ad’ as being infinite or nan. ( thats why i
think the pixels may be discarded )

similar problems arise as soon as I use f.i. the ‘dot’ function.

does anyone have an idea what’s going on ?

Thanks,

jonathan

I’m not sure that values can be “NAN” in GLSL, the GLSL spec states the result of some functions can be “undefined”.

Example: GLSL spec 1.20, page 57, section 8.2 Exponential Functions:

"
genType pow (genType x, genType y)

Returns x raised to the y power, i.e., x y
Results are undefined if x < 0.
Results are undefined if x = 0 and y <= 0.
"

I found this link once that illustrates the undefined behavior:

http://www.davidcornette.com/glsl/glsl.html

with this relevant section:
"A second pitfall to be aware of is that sometimes a variable may have an “undefined” value. Here is an example:

float x = pow(y, 2.0);
if (x >= 0.0 || x < 0.0) {
  // Set the fragment to be pure blue
  gl_FragColor = vec4(0.0, 0.0, 1.0, 1.0);
} else {
  // Set the fragment to be pure red
  gl_FragColor = vec4(1.0, 0.0, 0.0, 1.0);
}

You would expect the condition in the if statement to be true, and thus you would expect the fragment to be blue. However, if y is negative, the result of the call to pow() is undefined, so x is neither greater than, equal to, nor less than 0.0, so the fragment will actually be colored red.
"

Hi overlay,

Thanks for the reply, I’ve found this link too but it is
not what’s going on. all the values in my code seem valid
initially. it’s the moment that I use them in a function
like smoothstep, or dot that they become undefined, even
though the values are legal for that function…

J

by the way, if I change my textures to having GL_RGBA8 internal format rather than GL_RGBA32F_ARB, all is fine.

ok, it seems to have to do with the problem mentioned by
davidcornette after all; when i change line 3. to:

3.float ad = smoothstep( 0.0, advection + 0.00001,
clamp( p, 0.0, 1.0 ) );

all is fine. while the documentation to glsl states that the
result will be zero if x <= edge0, 1 if x >= edge1 and performs smooth Hermite interpolation between 0 and 1 when edge0 < x < edge1.

this does not seem to be the case, and the result seems to be in line with davids observations. Anyone have similar experiences ?

J

ok, solved. it’s the last line in the glsl language spec that gave me trouble:

smoothstep … Returns 0.0 if x <= edge0 and 1.0 if x >= edge1 and performs smooth Hermite interpolation between 0 and 1
when edge0 < x < edge1. This is useful in cases where
you would want a threshold function with a smooth
transition. This is equivalent to:
genType t;
t = clamp ((x – edge0) / (edge1 – edge0), 0, 1);
return t * t * (3 – 2 * t);
Results are undefined if edge0 >= edge1.

(pag. 58 )

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.