Linear texture interpolation in ARB_fp

If I have an 8-bit linear filtered texture that I am sampling from a fragment program (running in 24 bit precision), can I get out a color value that is not representable by an 8-bit number?

For example, if I had a texture that had rgb(0,0,232) and rgb(0,0,233) in adjacent texels and I sampled half way between them, could I get rgb(0,0,232.5) out? (I realize that they will be mapped to [0,1], but it’s easier to explain with ints).

Thanks,
Zeno

Yes. The value will be truncated to framebuffer precision in a stage after the fragment program execution.

Note that the framebuffer may be a floating-point framebuffer if you have set that up; in that case, truncation doesn’t have to happen (although clamping may happen, depending on set-up – usually, you don’t want this to happen when rendering to float).

I’m using the value in a calculation, not to directly apply a color to the screen. I should have asked if the register I assign the lookup to can have > 256 unique values.

I think your answer was yes, the texture linear interpolator operates at > 8 bit precision (even on an 8 bit texture) when using arb_fp, correct?

There are really two things to consider for your particular problem (probably in this order):

  • texture filtering precision
  • texture coordinate precision

Everybody does pretty high precision texture coordinates.

Texture filtering precision is probably 8-bit, so filtering between 232 and 233 will be effectively the same as nearest filtering.

On NVIDIA hardware, HILO textures (even HILO8 textures) are filtered at 16-bits, so you get a nice smooth transition between 232 and 233. You’ll have to do the filtering in your shader program to do better than that - until higher precision filtering hardware shows up.

Hope this helps…
Thanks -
Cass

That’s exactly the answer I was looking for. I was getting some artifacts while doing math in the fragment program that weren’t occuring when I did it on the CPU. An 8 bit filter explains it.

I’d like to use the HILO textures, but I’m running out of memory as-is, so my stuff will have to remain a pre-process. Thanks for the suggestion, though

– Zeno

Originally posted by Zeno:
[b]I’d like to use the HILO textures, but I’m running out of memory as-is, so my stuff will have to remain a pre-process. Thanks for the suggestion, though

– Zeno[/b]

HILO8 textures are only 8-bits per component. That shouldn’t cost you more memory than any other 8-bit-per-component texture.

Glad that helped…

Cass