View Full Version : A beginners guide to numerical precision

04-01-2003, 06:16 AM
Apologies to anyone actually looking for such a guide, since this is a request not a submission!

Given that, fancy new extensions excluded, according to the specs all pixel values are clamped to [0-1], what about internal computations using textures?

For example, say several textures are specified of type GL_FLOAT, contaning values from some arbitrary computation.
A fragment program is then used to combine values looked up from the textures, the output being written to the framebuffer, clamped to [0-1] as usual.
Provided that we're not upset about the final values being clamped, what kind of precision can we rely on internal to the fragment processor, when the textures are combined?

Hope you followed that.