Part of the Khronos Group
OpenGL.org

The Industry's Foundation for High Performance Graphics

from games to virtual reality, mobile phones to supercomputers

Results 1 to 10 of 15

Thread: Pixel Corruption From Neighboring Pixels with gl_FragCoord

Hybrid View

  1. #1
    Intern Contributor
    Join Date
    Sep 2007
    Location
    Southern California
    Posts
    60

    Pixel Corruption From Neighboring Pixels with gl_FragCoord

    I'm trying to create a color filter array demosaicing fragment shader for raw capture from a color camera with color bayer filter. In order to properly decode the image, I require acces to clean neighboring texels without any averaging from neighboring texels. Since I'm not using TEXTURE_RECTANGLE textures, I'm a little nervous. I also have to know the exact x/y offset of the main pixel in the shader.

    Here is a partial simple shader that implements the RGB values from a red pixel of the color bayer filter array:


    Code :
    [SIZE=2] #extension GL_EXT_gpu_shader4 : enable [SIZE=2][COLOR=#008000]//Required For Integer Modulus Operation (%)[/COLOR][/SIZE]
     
    uniform sampler2D BayerCaptureSampler;
    uniform [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]float[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] captureImageWidth;
    uniform [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]float[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] captureImageHeight;
     
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]void[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#000000] main([/COLOR][/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]void[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#000000])[/COLOR]
    {
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]float[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] step_w;
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]float[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] step_h;
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]float[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] fragOffsetFloatX;
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]float[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] fragOffsetFloatY
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]int[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] fragOffsetIntX;
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]int[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] fragOffsetIntY;
    vec2 redOffset[4];
    vec2 greenOffset[4];
    vec2 blueOffset[4];
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]int[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] evenOddX;
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]int[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] evenOddY;
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]float[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] Sum;[/SIZE][SIZE=2]
     
    [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Example: ASSUME ORIGINAL SOURCE BUFFER OF 2048 x 1536
     
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Calculate Single Texel Normalized Offset From Original capture buffer resolution
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]step_w = 1.0/captureImageWidth;           [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Should return 1.0/2048.0 = 0.00048828125
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]step_h = 1.0/captureImageHeigh;            [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Should return 1.0/1536.0 = 0.00065104166
     
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Retrieve the Current Texel Offset (In Host Buffer Integer Coordinates From Corner (0,0)
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]fragOffsetIntX = gl_FragCoord.x;             [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Should Return Integer Value 0 to 2047
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]fragOffsetIntY = gl_FragCoord.y;             [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Should Return Integer Value 0 to 1535
     
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Determine if this is an even or odd pixel on horizontal and vertical offsets from bottom right corner texel
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]evenOddX = fragOffsetIntX%2;               [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Should Return 0 for even, or 1 for odd
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]evenOddY = fagOffsetIntY%2;                [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Should Return 0 for even, or 1 for odd
     
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//*************************Sample Code For A Red Pixel Decocing******************//
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]if[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2](evenOddX==0) [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Width Offset = Even Texel
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]{
    [/SIZE][SIZE=2][COLOR=#0000ff][SIZE=2][COLOR=#0000ff]if[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] {evenOddY==0) [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Height Offset = Even Texel
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]{
    [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Retrieve Red Texel
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]glFragColor.r = texture2D(BayerCaptureSampler,vec(gl_TextCoord[0].st)).r;
    [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Calculate Green Pixel Offsets
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//THESE NEED TO BE IDENTICAL PIXEL VALUE FROM THE ORIGINAL BUFFER THAT WAS LOADED INTO
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//TEXTURE WIHTOUT ANY MIXTURE OR AVERAGING OF PIXELS ONCE INSIDE THE TEXTURE
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] greenOffset[0] = vec2(step_w,step_h);
      greenOffset[1] = vec2(0.0,step_h);
      greenOffset[2] = vec2(-step_w,0.0);
      greenOffset[3] = vec2(0.0,-step_h);
    [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Retrieve and Average Neighboring Green Pixels
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] sum = 0.0;
      sum += texture2D(BayerCaptureSampler,vec2(gl_TexCoord[0].st + greenOffset[0];
      sum += texture2D(BayerCaptureSampler,vec2(gl_TexCoord[0].st + greenOffset[1];
      sum += texture2D(BayerCaptureSampler,vec2(gl_TexCoord[0].st + greenOffset[2];
      sum += texture2D(BayerCaptureSampler,vec2(gl_TexCoord[0].st + greenOffset[3];
    [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Assign Final Calculated Green Texel
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]glFragColor.g = sum/4.0;
    [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Calculate Blue Pixel Offsets
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//THESE NEED TO BE IDENTICAL PIXEL VALUE AS DESCRIBED ABOVE
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] greenOffset[0] = vec2(step_w,step_h);
      greenOffset[1] = vec2(-step_w,step_h);
      greenOffset[2] = vec2(step_w,-step_h);
      greenOffset[3] = vec2(-step_w,-step_h);
    [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Retrieve and Average Neighboring Blue Pixels
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] sum = 0.0;
      sum += texture2D(BayerCaptureSampler,vec2(gl_TexCoord[0].st + blueOffset[0];
      sum += texture2D(BayerCaptureSampler,vec2(gl_TexCoord[0].st + blueOffset[1];
      sum += texture2D(BayerCaptureSampler,vec2(gl_TexCoord[0].st + blueOffset[2];
      sum += texture2D(BayerCaptureSampler,vec2(gl_TexCoord[0].st + blueOffset[3];
    [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Assign Final Calculated Blue Texel
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]glFragColor.b = sum/4.0;
       }
    }
    [/SIZE][SIZE=2][COLOR=#008000][SIZE=2][COLOR=#008000]//Assign 1 to Alpha Channel
    [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]glFragColor.a = 1.0
    }
    [/SIZE]

    My questions are:

    1) How do I implement the layout qualifier "pixel_center_integer" with this version of OpenGL? I need the offsets to be the actual integer values, not with 0.5 offsets.

    2) Will my greenOffset[] and blueOffset[] texel offsets return me the original buffer pixels without any sub-pixel neighbor averaging? The color demosaicing will not work if the pixel values are in any way mixed.


    Thanks for your help!
    Last edited by Rennie Johnson; 10-05-2012 at 04:53 PM.

  2. #2
    Senior Member OpenGL Guru Dark Photon's Avatar
    Join Date
    Oct 2004
    Location
    Druidia
    Posts
    3,213
    Quote Originally Posted by Rennie Johnson View Post
    1) How do I implement the layout qualifier "pixel_center_integer" with this version of OpenGL? I need the offsets to be the actual integer values, not with 0.5 offsets.

    2) Will my greenOffset[] and blueOffset[] texel offsets return me the original buffer pixels without any sub-pixel neighbor averaging? The color demosaicing will not work if the pixel values are in any way mixed.
    Often a simpler way to deal with this is to just use glFragCoord and texelFetch.

    The integer offset of the current pixel (in window coordinates):

    ivec2 tcoord = ivec2( gl_FragCoord.xy ); // This is in (0,0) .. (xres,yres)

    then:

    texelFetch( tex, tcoord, 0 ).rgb;

    If you're reading from a multisample texture, you can just plug the sample index in place of the third argument.

    Also with texelFetch, there's explicitly no filtering, so you don't have to worry about that creeping in.
    Last edited by Dark Photon; 10-05-2012 at 06:22 PM.

  3. #3
    Intern Contributor
    Join Date
    Sep 2007
    Location
    Southern California
    Posts
    60
    Hey, thanks Dark Photon!!

    Do I need to declare a higher version of OpenGL or an extension to use "texelFetch"?

    If so, could you type out the line for me?

    Also, since you still used glFragCoord(), don't I still have to deal with the 0.5 pixel offset?

    Thanks again.

  4. #4
    Intern Contributor
    Join Date
    Sep 2007
    Location
    Southern California
    Posts
    60
    I was just reading up on textlFetch. How do I specify the offsets for my neighboring pixel access?

    Can I use integer offsets, i.e.:

    ivec2 offset;
    offset = ivec2(1,-1);
    ivec2 tcoord = ivec2( gl_FragCoord.xy );
    texelFetch(TruesenseCaptureSampler, (tcoord + offset), 0 ).r;


    or do I have to use Normalized offsets, like in my code example;

    Again, do I have to implement the layout qualifier "pixel_center_integer" in this example, and what is the correct syntax to do that?

    Thanks for your help.

    p.s. Can someone explain how to post code on a reply? I can't find the box to do that.

  5. #5
    Senior Member OpenGL Guru Dark Photon's Avatar
    Join Date
    Oct 2004
    Location
    Druidia
    Posts
    3,213
    Quote Originally Posted by Rennie Johnson View Post
    I was just reading up on textlFetch. How do I specify the offsets for my neighboring pixel access? Can I use integer offsets)?
    Yes! You've got it. That's one of the nice things about using this for single texel access.

    Again, do I have to implement the layout qualifier "pixel_center_integer" in this example, and what is the correct syntax to do that?
    No idea what you're talking about here. This does not occur in your code. If you want no interpolation on interpolators, declare them flat. You have to do this when passing integral values anyway.

    p.s. Can someone explain how to post code on a reply? I can't find the box to do that.
    Surround your code with [code] ... [/code] tags.

  6. #6
    Intern Contributor
    Join Date
    Sep 2007
    Location
    Southern California
    Posts
    60
    OK, I'll try all this and report back.

    Seems a bit rude to whack fractions, but I guess I've got no choice.

    Thanks again to Dark Photon.

    p.s. I'm not sure what the origin of your profile photo is (looks like a cross between Darth Vader and a Dalek, but you might be interested to know I that attended the first showing of STAR WARS (1977) in San Jose. It was the greatest experience (female company excluded) of my life!

  7. #7
    Senior Member OpenGL Guru Dark Photon's Avatar
    Join Date
    Oct 2004
    Location
    Druidia
    Posts
    3,213
    Quote Originally Posted by Rennie Johnson View Post
    Hey, thanks Dark Photon!!

    Do I need to declare a higher version of OpenGL or an extension to use "texelFetch"?
    Depends on which version you are declaring now. May already be supported. Heck, it's been in GLSL since v1.3. So anything greater than or equal to #version 130 should pull it in.

    Alternatively, you'll probably just get it with the #extension GL_EXT_gpu_shader4 : enable you've got in there now, though you'll likely need to call texelFetch2D() instead of texelFetch() (for 2D textures anyway). GLSL 1.3 did away with all the sampler type suffixes from the names of the sampling functions, eliminating a lot of needless function list bloat.

    The issue with declaring 1.3 right now if you're using 1.2 (the default IIRC) is that the built-in identifiers (fixed-function shader shims) go away. Which may or may not be more code rework than you want to bite off right now.

    Also, since you still used glFragCoord(), don't I still have to deal with the 0.5 pixel offset?
    I cast to an int (ivec2), which (just as in C/C++) whacks the fractional portion.
    Last edited by Dark Photon; 10-06-2012 at 11:10 AM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •