Floating point accuracy in GLSL

I am working with uint8 and uint16 textures in GLSL ES.

What really happens when
float X = texture2D(TestTex, vec2(0.0,0.0) ).x;is called?

Let’s say uint8 34 is stored at texel (0,0). Am I correct the above is equivalent to:
float X = float(34)/float(255);

But, the cast from uint8 to float introduces error, and so does the division.

So if I want to get 34 back exactly in my shader, can I trust either of

int X1 = round(X255.0);
int X2 = int(X
255.0);

?

And what’s this thing about GLSL being IEEE754 like? How different is it in a nutshell?

Thanks

If you are using integer texture format, not a fixed point normalized texture format, you can directly do the appropriate integer texture fetches using integer samplers (usampler* or isampler*, the former could come handy in your case) as that way you get the actual integer texel values without the need of ALU instructions to scale it back up.

If you do use normalized texture formats (maybe because you are using a GPU generation that does not support integer texture formats), then your idea may work as well, however AFAIK there is no absolute guarantee about precision for pre-GL3 hardware.

But, the cast from uint8 to float introduces error
No error is introduced casting ints or uints to floats unless an int’s or uint’s magnitude exceeds the maximum value representable with the number of bits in the float’s mantissa. No uint8 can ever do that, even for a float16.

For GLSL ES there are no integer samplers AFAIK. Is it possible to use them via any extensions for GL3 capable hadrware?

Ah, sorry, didn’t realize you look a solution for ES.
But yes, it is possible to use integer textures on GL3 capable hardware with GL_EXT_texture_integer that is already a core extension.

[b]
How does this work?

A way to find the mantissa and exponent from an int is:
log2(34) ~= 5.0875
exponent = ceil(5.0875) = 6
mantissa = 34 / ( 2^(exponent) ) ~= 0.5313If I map back I get
0.5313 * 2^6 = 34.0032

Does IEEE754 use other method much more accurate or exactly accurate? [/b]

Also
http://wiki.flightgear.org/index.php/Howto:_Shader_Programming_in_FlightGear
says GLSL float is ‘IEEE754 like’. What does that mean?

Thanks

How does this work?

A way to find the mantissa and exponent from an int is:
log2(34) ~= 5.0875
exponent = ceil(5.0875) = 6
mantissa = 34 / ( 2^(exponent) ) ~= 0.5313If I map back I get
0.5313 * 2^6 = 34.0032

Does IEEE754 use other method much more accurate or exactly accurate?

Yes, IEEE 754 uses another, exactly accurate (and faster and simpler) method. In most cases, IEEE 754 floating point numbers are encoded in normalized form. The log2 value of a number is never calculated, making the normalized encoding faster, simpler, and exact (as long as the magnitude of the number fits within the mantissa’s bits).

IEEE 754 normalization means that the binary point position of a value is shifted one bit (and the exponent incremented or decremented by one) until the absolute value of the value is one or between one and two (except for the special values of zero, infinity, NaN, etc.). The exponent indicates how many positions the binary point was moved and the mantissa is not changed (except that the leading bit, which after normalization will be one, is removed to allow an extra bit of precision in the least-significant bit of the mantissa).

So, in your example, the binary value of 34 is 100010. Normalized, that is represented as 1.00010 with an exponent of five (added to a bias to allow both positive and negative exponents). After the leading one is removed the mantissa will be .00010 with however many trailing zeroes as are required to fill the remaining less significant bits of the mantissa for the particular floating point size (16, 32, 64, 80 bits).

Reconstructing the value, you concatenate the implicit leading one to the mantissa, for 1.00010, then shift the binary point five bits to the right from its position to the right of the leading one, for a value of 100010.0, which equals 34 exactly.

This can be implemented in hardware very efficiently and is very fast.

Also
http://wiki.flightgear.org/index.php/Howto:_Shader_Programming_in_FlightGear
says GLSL float is ‘IEEE754 like’. What does that mean?

Well, the GLSL 4.0 spec says this about floats (in section 4.1.4):

So, to answer your question, IEEE 754-like in GLSL pretty much means that the special encoding and processing for the IEEE 754 special values NaN, plus infinity, and minus infinity isn’t required.

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.