Heavy rounding in RGB16 ?

Hello,

I’m drawing to a pbuffer on a Radeon 9800 Pro, Catalyst 3.4 drivers.

I have a 16-bit RGB mode set (all R,G,B channels are 16-bit).

It seems some very heavy rounding is done when using this mode.

Example:

Rendering a GL_FLAT -shaded quad with the color glColor3us(0xABCD, 0, 0) results in a color that is rounded to 0xABBF.

Now, before you fire up your calculators here are the bits:

0xABCD: 1010 1011 1100 1101
0xABBF: 1010 1011 1011 1111

That’s 7 bits rounded, almost HALF of the bits! Is the rounding really this heavy? How could I prevent this? I tried some other glColor??() functions but they go round and round and round… ;(

I’m very disappointed with this, as I’m trying to do scientific calculation here using pbuffers - whats the point in switching to a 16-bit mode if the accuracy is only half the bits ?

Any ideas on how to improve accuracy?

Thanks for any input,

Andru

RGB16 is a floating point format right? It is probably impossible to represent the integer 0xABCD as a 16 bit floating point number.

There are not 16-bits of precision in the RGB16 format, some of those bits are used for the sign and exponent. The whole point is to have greater range (and I think, greater precision after doing multiple calculations, a case were fixed point would fail without extra effort).

[This message has been edited by Nakoruru (edited 06-16-2003).]

No I don’t think that’s a float format, that should be a totally different issue. At least I haven’t used any pixel_format_float extensions in my code

Anyway, the output seems like the values are converted to a (16-bit) float value at some point, which is a great shame in this case.

Andru

The colors interpolate with a bit less precision than the texture coordinates. I would suggest trying to pack them in a texture coordinate and using a pixel shader to pass them to the output.

BTW, I would warn you that you might see this issue other places, since the GL spec explicitly allows lower precision and range on color values than texture coordinates.

-Evan

Originally posted by ehart:
[b]
The colors interpolate with a bit less precision than the texture coordinates. I would suggest trying to pack them in a texture coordinate and using a pixel shader to pass them to the output.

BTW, I would warn you that you might see this issue other places, since the GL spec explicitly allows lower precision and range on color values than texture coordinates.

-Evan[/b]

Evan,

This is not a matter of interpolation - the object in question was a single GL_QUAD with the GL_FLAT shading model, thus no interpolation…(?)

I see what you mean though, and I think I’m going to switch over to a 32-bit Red channel in hope for a bit more precision.

Thanks for the answers, they cleared things up a bit.

Andru

It seems the mode I am setting for my pbuffer (RGB16 mode), is floating point.

How can I set or even query for a non-floating point mode ? Neither WGL_ARB_pbuffer nor WGL_ARB_pixel_format say anything about the formats being float or fixed-point (?)

Do I have to iterate through all suitable modes and check which ones are fixed-point? This would be a very kludgy solution… I am assuming now that fixed-point 16-bit modes (R16G16B16xxx) or higher exist.

I need a fixed-point mode to enable blending. Blending doesn’t work in float modes it seems.

Thanks for any help,

Andru

[This message has been edited by Andru (edited 06-17-2003).]

First, when I said interpolator, I just meant the HW path that moves the color from the vertex level to the fragment level. The comment about the color attribute having less precision still holds.

Next, unless you request a floating point buffer, you should get an int buffer. Unfortunately, blending is still not supported in HW for RGBA16 surfaces.

-Evan

I was under the impression that particular piece of hardware could only support 12 bits per channel max. And if you spread those 12 bits over the domain of 16 bits and do the math, you will come to an interesting conclusion.

[This message has been edited by DFrey (edited 06-17-2003).]

The R3xx chips are full 24bit float in the pipeline, it also supports 16bit/channel fixed point textures/pbuffers.

You’re probably thinking of the 8500 (R200)

[This message has been edited by NitroGL (edited 06-17-2003).]

How many bits does the gamma ramp support per channel?

Originally posted by DFrey:
How many bits does the gamma ramp support per channel?
You cannot tell without detailed analog measurement.

Win32 exposes gamma ramps as 16 bits unsigned.

But (coming straight to the point ), the gamma ramp is irrelevant for all things rendering. It only applies to RAMDAC operation. If you’d like to convince me of the opposite, go ahead

Nope, not gonna try to convince you otherwise, as obviously that makes sense. I was half asleep when I presented that question.