PDA

View Full Version : Correct hdr downsampling with fragment shader and texturefilter



namespace
07-29-2006, 11:26 AM
Hi!

I'm implementing a HDR renderpath using framebuffer objects and GLSL. I want to add effects like tonemapping and glow so I need to create a downsampled image of my scene to do blurring etc.

I downsample the scene to a 1/4 of its size with an extra shader which reads and averages a 4x4 box of texels.

Since modern cards support bilinear filtering for hdr textures (I use the RGBA16F format from the arb extension) I want to write a second shader which does 4 texture lookups instead of
16.

I want to place the 4 sampling points in the edges of 4 texels:

http://www.codecreator.net/screens/downsampling.png
(the blue pixel is the "current" one, red marks the sampling points)

Which gives me the following code in my vertexshader:

gl_TexCoord[0].xy = gl_MultiTexCoord0.xy + vec2(0.5, 0.5) * texel_size;
gl_TexCoord[1].xy = gl_MultiTexCoord0.xy + vec2(0.5, 2.5) * texel_size;
gl_TexCoord[2].xy = gl_MultiTexCoord0.xy + vec2(2.5, 2.5) * texel_size;
gl_TexCoord[3].xy = gl_MultiTexCoord0.xy + vec2(2.5, 0.5) * texel_size;However this does not yield the same results as the 16-lookup shader although I set the texture to GL_LINEAR/GL_LINEAR for the 4-lookup shader and GL_NEAREST,GL_NEAREST for the 16-lookup shader.

After some exausting trial and error I've found the correct code:


gl_TexCoord[0].xy = gl_MultiTexCoord0.xy + vec2(0.0, 0.0) * texel_size;
gl_TexCoord[1].xy = gl_MultiTexCoord0.xy + vec2(0.0, 2.0) * texel_size;
gl_TexCoord[2].xy = gl_MultiTexCoord0.xy + vec2(2.0, 2.0) * texel_size;
gl_TexCoord[3].xy = gl_MultiTexCoord0.xy + vec2(2.0, 0.0) * texel_size;This makes no sense to me.
My understanding of texturing is that opengl samples from the
center of the texel where as directx samples from the lower left corner. So this offsets should work with directx but not opengl.

I read the spec and can't find my mistake, I guess its simple and i just don't see it.
Please help!

This image show the delta between the 16 and 4-lookup shader.

http://www.codecreator.net/screens/diff.jpg

k_szczech
07-29-2006, 05:47 PM
I'm using the 0.5 / 2.5 offset myself. I'm not passing texel size to a shader - I just pass all 4 texture coordinates directly from application. Here is my code:

fog::vec2i targetSize = fog::Display::currentMode.size;
fog::vec2f lastTexel((float)targetSize.x / (float)hdrSize.x,
(float)targetSize.y / (float)hdrSize.y);
fog::vec2f texelOffset(1.0f / (float)hdrSize.x,
1.0f / (float)hdrSize.y);
glBegin(GL_QUADS);
glMultiTexCoord2f(GL_TEXTURE0_ARB, 0.5f * texelOffset.x, lastTexel.y + 0.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE1_ARB, 2.5f * texelOffset.x, lastTexel.y + 0.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE2_ARB, 0.5f * texelOffset.x, lastTexel.y + 2.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE3_ARB, 2.5f * texelOffset.x, lastTexel.y + 2.5f * texelOffset.y);
glVertex2f(0.0f, 0.0f);
glMultiTexCoord2f(GL_TEXTURE0_ARB, lastTexel.x + 0.5f * texelOffset.x, lastTexel.y + 0.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE1_ARB, lastTexel.x + 2.5f * texelOffset.x, lastTexel.y + 0.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE2_ARB, lastTexel.x + 0.5f * texelOffset.x, lastTexel.y + 2.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE3_ARB, lastTexel.x + 2.5f * texelOffset.x, lastTexel.y + 2.5f * texelOffset.y);
glVertex2f((float)targetSize.x, 0.0f);
glMultiTexCoord2f(GL_TEXTURE0_ARB, lastTexel.x + 0.5f * texelOffset.x, 0.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE1_ARB, lastTexel.x + 2.5f * texelOffset.x, 0.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE2_ARB, lastTexel.x + 0.5f * texelOffset.x, 2.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE3_ARB, lastTexel.x + 2.5f * texelOffset.x, 2.5f * texelOffset.y);
glVertex2f((float)targetSize.x, (float)targetSize.y);
glMultiTexCoord2f(GL_TEXTURE0_ARB, 0.5f * texelOffset.x, 0.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE1_ARB, 2.5f * texelOffset.x, 0.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE2_ARB, 0.5f * texelOffset.x, 2.5f * texelOffset.y);
glMultiTexCoord2f(GL_TEXTURE3_ARB, 2.5f * texelOffset.x, 2.5f * texelOffset.y);
glVertex2f(0.0f, (float)targetSize.y);
glEnd();targetSize is the screen size
hdrSize is the size of texture used for HDR (I'm using power of two textures, so screen is smaller than texture)
lastTexel is one after the last texel in the HDR texture that is actually used.
Example:
targetSize = (1280, 1024)
hdrSize = (2048.0, 1024.0)
lastTexel = (0.625, 1.0)
If you're using entire texture then texel size will be (1.0, 1.0).

Make sure that your application is not messing up things (passing wrong coordinates, texgens, matrices)

namespace
07-30-2006, 08:43 AM
I'm pretty sure I do everything the correct way:


(Code in Renderer)

glserver->SetFrameBuffer(*fb_downsample.get());
glserver->SetFrameBufferTargetColor(0, rt_downsample.GetTexture());
glserver->SetFrameBufferTargetDepthNone();

if(!glserver->CheckFrameBufferCompleteness())
{
runl << "FB INC" << endl;
return;
}

glserver->SetMatrixMode(GL_PROJECTION);
glserver->PushMatrix();
glserver->ResetMatrix();
glOrtho(0.0f, rt_downsample.utex_size[0], 0.0f, rt_downsample.utex_size[1], -1.0f, 5.0f);
glserver->SetMatrixMode(GL_MODELVIEW);
glserver->PushMatrix();
glserver->ResetMatrix();

glserver->SetViewport(0, 0, rt_downsample.utex_size[0], rt_downsample.utex_size[1]);

{ // downsample
glserver->SetMaterial_Downsample(rt_scene.GetTexture(), f_float_bilinearfilter);
Quad(rt_downsample.frt_size[0], rt_downsample.frt_size[1], rt_scene.st[0], rt_scene.st[1]);
}



(Code in backend)

void glserver_glsl::SetMaterial_Downsample(texture_2d&amp; tex, bool use_bilinear_filter)
{
shader_sampler_glsl* sh = (use_bilinear_filter) ? sh_downsample_bilinear.get() : sh_downsample_nearest.get();
SwitchShader(*sh);

BindTextureQuick(0, tex);

float texel_size[] = { 1.0f / (float)tex.GetWidth(), 1.0f / (float)tex.GetHeight() };
sh->MS_TexelSize(texel_size);
}


and

void shader_sampler_glsl::MS_TexelSize(const float ts[2]) { glMultiTexCoord2fv(GL_TEXTURE2, ts); }
frt_size is the size of the output image (ie 1024x768 or someting like that)
utex_size is the size of the texture (1024x1024)
and st are frt_size / utex_size , or last_texel as you call it.

namespace
07-30-2006, 02:26 PM
Ok i made second test and passed texel_size as a uniform to the shader. The filtering is still "wrong".

Edit:
k_szczech, what hardware do you use?
My card is a geforce 6. Maybe its a driver issue.

k_szczech
07-30-2006, 06:04 PM
Guess what - I've just tested my code and it turns out I'm having exactly the same thing you have - I need to use 0.0 / 2.0 and not 0.5 / 2.5 to get proper filtering.

Ok, here is what happens:
Rendering 1024x768 texture to 1024x768 viewport:
Every pixel on screen is one texel on texture:
http://ks-media.ehost.pl/opengl_org/sample1x1.png
Red dot is the sampling point.

Rendering 1024x768 texture to 256x192 viewport (4x downsample):
a grid of 4x4 texels fills every pixel on screen:
http://ks-media.ehost.pl/opengl_org/sample4x4.png

So proper offsets are:
-1.0 * texel_size
+1.0 * texel_size

Turns out your post helped me to realize that I have something wrong with my code. In return I gave you the solution. Now that's a teamwork :D

namespace
07-31-2006, 03:22 AM
Nice, thanks alot. :)

Now I don't have this strange feeling anymore
about code that works although i don't know why ;)

ebray99
07-31-2006, 09:22 AM
It's probably because your quad is not rasterized directly on pixel centers at the edge, but rather on a pixel borders. The interpolated texture coordinate would then be offset by approximately 0.5*texel in the fragment shader.

Kevin B

Relic
07-31-2006, 09:37 AM
It's probably because your quad is not rasterized directly on pixel centers at the edge, but rather on a pixel borders. The interpolated texture coordinate would then be offset by approximately 0.5*texel in the fragment shader.
That's exactly the opposite of what's needed.
The ortho matrix and the vertex coordinates for pixel operations need to be exactly on the pixel's corners, so that the texture coordinate interpolation inside the fragment pipeline takes care to hit the centers of the texels in a 1-to-1 mapping.

ebray99
07-31-2006, 11:44 AM
That's exactly the opposite of what's needed.
The ortho matrix and the vertex coordinates for pixel operations need to be exactly on the pixel's corners, so that the texture coordinate interpolation inside the fragment pipeline takes care to hit the centers of the texels in a 1-to-1 mapping.
namespace simply stated he didn't know why the offsets weren't working the way he thought they should. I was simply explaining what *is* happening, not what he should do (you basically said what I was trying to say). If only we could draw an image on this forum, then explaining details like this would be much easier. =\

Kevin B

Relic
08-01-2006, 01:03 AM
I think we all agree how it needs to be setup. I just wanted to make sure everybody else gets this right in OpenGL.
Too bad that DX9 <ducks> got that totally wrong. ;) For your confusion:
http://msdn.microsoft.com/library/defaul...s_to_pixels.asp (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directly_mapping_texels_to_pixels.asp)

k_szczech
08-01-2006, 02:01 AM
:D
I see every opportunity to complain about DX is good on OpenGL forums. :)

I was simply explaining what *is* happening (...) If only we could draw an imageI have explained what's going on and I've used images to do so. Is my previous post invisible? ;)
Just to avoid confusion: this topic was about downsampling image 4 times in single pass and not about 1:1 mapping.

To put image in your post you must host it on external server.
Cheers!

namespace
08-01-2006, 05:28 AM
For your confusion:
http://msdn.microsoft.com/library/defaul...s_to_pixels.asp (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/directx9_c/directly_mapping_texels_to_pixels.asp)
Works, I'm confused now ;)


Well I think thats the reason why our DX buddies have to implement an extra upsampling shader:
<from the bloom example by nvidia>


VS_OUTPUT VS_Quad(float4 Position : POSITION,
float2 TexCoord : TEXCOORD0)
{
VS_OUTPUT OUT;
float2 texelSize = 1.0 / WindowSize;
OUT.Position = Position;
// don't want bilinear filtering on original scene:
OUT.TexCoord0 = TexCoord + texelSize*0.5;
return OUT;
}
Crazy...