Multisample resolve is not gamma correct with FBOs

Hi,

I tested this with both Nvidia GT200 and G80/G90 series (drivers release Forceware 177.35 & 177.41). If the rendering is done without FBOs i.e. multisampling is controlled either by the Nvidia control panel or by selecting a pixel format with a multisample buffer (ARB_multisample extension) then the AA resolve is gamma correct. The Nvidia control panel parameter “Antialiasing - Gamma correction” must be set to “on” though.

Gamma correct means each MSAA sample is de-gammatized prior resolve and then gammatized again. Gamma correct resolve results in a much better AA quality as shown by the following screenshots of my test application (@ 8xQ - 8x pure MSAA, no CSAA):

Gamma correct star (no FBOs):
http://cdn-2-service.phanfare.com/images…89b9628ae6e92_5

Gamma incorrect star (FBOs):
http://cdn-2-service.phanfare.com/images…fac5ab20f8883_5

The gamma incorrect version looks like being outlined black…

The theory behind this is well explained here:
http://www.all-in-one.ee/~dersch/gamma/gamma.html

In addition, here is a test that “magnify” the MSAA gradient seen on the edge of a white triangle on a black background. The middle gradient band is the MSAA gradient, the upper band the expected gamma correct resolve result and the bottom band is the expected result without gamma correction.

Gamma correct gradient (no FBOs):
http://cdn-2-service.phanfare.com/images…02f368af83ae3_5

Gamma incorrect gradient (FBOs):
http://cdn-2-service.phanfare.com/images…c7b6731ecf11d_5

From the following specification note it seems all the API is there…

Excerpt from “EXT_framebuffer_sRGB”:

"17) How does this extension interact with multisampling?

    RESOLVED:  There are no explicit interactions.  However, arguably
    if the color samples for multisampling are sRGB encoded, the
    samples should be linearized before being "resolved" for display
    and then recoverted to sRGB if the output device expects sRGB
    encoded color components.

    This is really a video scan-out issue and beyond the scope
    of this extension which is focused on the rendering issues.
    However some implementation advice is provided:

    The implementation sufficiently aware of the gamma correction
    configured for the display device could decide to perform an
    sRGB-correct multisample resolve.  Whether this occurs or not
    could be determined by a control panel setting or inferred by
    the application's use of this extension."

As a result I would expect this to work in order to get a gamma correct AA resolve:

- Render to FBO with a render buffer's internal format set to GL_SRGB8_ALPHA8_EXT (EXT_texture_sRGB - not supported for now, FBO not complete reported);
- Enable GL_FRAMEBUFFER_SRGB_EXT for the destination FBO (EXT_framebuffer_sRGB);
- Perform blit/resolve function.

However it would be nice to be able to render to an higher precision formats and still be get gamma correct AA (such as RGB10_A2, R11F_G11F_B10F_EXT or GL_RGBA_FLOAT16). So it could work like this:

- Enable GL_FRAMEBUFFER_SRGB_EXT for the source FBO (EXT_framebuffer_sRGB);
- Enable GL_FRAMEBUFFER_SRGB_EXT for the destination FBO (EXT_framebuffer_sRGB);
- Perform blit/resolve function.

Is there a plan to fix this for Nvidia? Is it working on other vendors implementations?

thanks,

Cub

Interesting observation, but the behaviour does not necessarily sound like an error to me.

Gamma-correction compensates for non-linearity of the display monitor and perhaps even for the user’s eyes, so should be the very last step before the user sees the results. I would assume that gamma correction by default should not apply to intermediate results.
Since you have observed that the results are correct when stuff is being rendered to the frame buffer, and not when rendered to textures, it sounds to me like this is correct behaviour.

I can however imagine incorrect results if gamma correction is applied to the fragment output before it’s written to the multisample buffer.
Gamma correction should in my opinion be applied only to the final results in the framebuffer itself. (Which is where I disagree with some of nVidia’s recommendations in the “importance of being linear” chapter in GPU GEMS 3, which suggests putting gamma correction in the fragment shader).

There are two completely separate concepts:

  • Correcting for the non-linear output response curve of the display device, and
  • Storing colour information in non-linear format and linearising it before performing computations.

The former should indeed only happen in the display pipeline which sends the image to the monitor. But the latter is effectively a form of colour compression and should be performed any time colour information is written to or read from an image marked as sRGB, including texture fetch, blending, framebuffer writes, and downsampling.

It’s a shame sRGB storage is so often referred to as “gamma correction” since that’s not what it is. It’s a fixed encoding that takes the differences in perception of dark and light colours into account, and it only makes sense for fixed point formats such as RGB8.

R11F_G11F_B10F_EXT or GL_RGBA_FLOAT16 should be used to store linear colours since the float representation already guarantees much higher precision for dark colours.

Confusing terminology aside, and assuming that the FBO used by Cubitus is indeed in sRGB format, the issue seems to be that the raw values are being interpolated linearly instead of being decoded first, as is probably the case with the frame buffer.

Then I would concur with Cubitus’ assessment that this is a bug. Might not be a driver-level bug though, it may well be fixed functionality in hardware.

I agree that this might not be a bug. However the API should let the application specify if linearization is needed prior MSAA samples averaging, and before storing it if the destination buffer has limited precision (fix point 8-bit or 10-bit per component). In order to keep precision when it matter the most…

Note 1: On a GeForce card with release 177.35 & 177.41 the FBO resolve operation behave like it is without FBO if the blit/resolve destination FBO is 0 (backbuffer) i.e. linearization occur before average when the Nvidia control panel parameter “Antialiasing - Gamma correction” is set to “on”. This is not a great workaround since an ugly glCopyTexImage is then needed to get back the result to texture, the FBO size is limited to window size and formats are limited to pixel formats.

Note 2: The internal format of the render buffer is indeed GL_RGBA8 since GL_SRGB8_ALPHA8 is not allowed (FBO not complete).

Cub

Correction: It is allowed to render to a renderbuffer with internal format set to GL_SRGB8_ALPHA8 (I did a mistake in my test) but still it does not change the end result: the AA resolve is not gamma correct.

Cub

It is 2012 and this problem persists. It is something that (if it only ever crops up as a result of resolving multisampled FBOs) is very difficult to perceive unless one is carefully looking for it.

Well, that’s exactly what I did today. I was very very carefully examining why the FBO multisampled results are ever so slightly different between my two techniques.

My system is a Nvidia GTX 260 on driver 285.62 on Windows 7 SP1.

In one mode I render a scene to a multisampled FBO and blit it directly to the backbuffer.

In the other, I render the same scene, blit it to a texture, and render that texture.

By binding my mouse wheel to flip between the two display modes, I noticed with 8-sample multisampling that my triangles look ever so slightly shrunken using the second method.

I then went on to use glReadPixels to extract pixel values from both the front buffer and the FBO buffers:

Pixel location (162,312)
Front buffer: R=34 G=34 B=99
Attachment0: R=12 G=12 B=32

The geometry I drew was colored with the color R=100 G=100 B=255.

Here, it seems that I have isolated a pixel which had a coverage of 1/8, since 256/8 is 32. It’s reasonable to assume therefore that without “gamma correction” the resolve operation simply applies the color linearly scaled with the coverage value.

I assumed up to this point that this method would be correct, but as the gamma-upscaled image looks more correct to me than the one that is linearly scaled with coverage, In particular, on edges that are very nearly aligned with the pixel grid, with the non-gamma-corrected image I get the visual effect of gentle “notches” along the edge.

So what is it about the way we perceive light that makes covering 1/8 of a pixel require changing its value by over 1/3 in order to get it to look right?

I understand that we can detect change in light level better among the high range of our monitors’ color levels than in the low range. I’m confused about where in the graphics pipeline we ought to be applying the correction.

Either way I’d like to get a proper fix for this issue sorted out. What the “bug” effectively does is it makes low-coverage pixels much less visible than they should be: This makes the antialiasing effect less than ideal.

But on the other hand, if an algorithm was to use multisampling and assumes the output is a linear mix of the individual sample values (which is not at all unreasonable, though I have a very difficult time imagining a GPGPU application making use of multisampling), performing the correction may be incorrect in those cases.

I think the right way to deal with this is to provide a way to specify if correction is to be applied during the blit operation. Perhaps introduce an alternative to glBlitFramebuffer, perhaps glBlitFramebufferMS.

But, what’s more important than that is the question of how to control this behavior. How can I get the gamma-correct resolve, i.e. without the visual result of having a dark outline, without first blitting to the backbuffer?

It is 2012 and you’re supposed to use texelFetch(sampler2DMS) :slight_smile: . Can do exposure control there, too - as you should; and thus use at least R10G11B10F.
You’re given control, use it.

Fair enough…

I’m trying to produce code which I intend to transition to GL ES 2 and WebGL eventually.

Sampler2DMS gives me control over the entire resolve process, which is quite good obviously.

To recap, the correct way to deal with this issue on RGB8 is to use ARB_framebuffer_sRGB.

OpenGL is only responsible for performing calculations on a linear scale. The gamma correction need only be applied once, at the very end of the rendering process.

Therefore the multisample resolve which linearly combines samples is the correct way to resolve.

My understanding is that display hardware expects a gamma-corrected signal: With a 0.5 input value (a value of 128 with today’s 24bit displays) a 22% light intensity is to be produced. As for using higher precision pixel formats, they are only useful for providing more precision while working in the linear color space, and the resulting frames need to be gamma-corrected prior to output.

Please correct me if I’ve got this wrong. While using an sRGB framebuffer, the values are stored in the sRGB space itself, while all operations involved will be performed in the correct corresponding linear space. This produces higher quality than a manual method of using a regular linear color buffer then gamma-correcting (with a shader) in post processing. Because the sRGB framebuffer linear operations will perform in full precision in the hardware, while the manual method has it truncated to 8 linear bits at every pass.

Here is some reading material that is related to this topic. http://scripts.breidt.net/gamma_correct_v12.pdf
http://http.developer.nvidia.com/GPUGems3/gpugems3_ch24.html