Fullscreen quad invoking fragment shader twice along triangle seam?

I’m trying to implement linked-list based transparency. The final stage is to performs alpha blending on a linked list of fragments for each pixel. The problem I’m having is that, when I try to render a fullscreen quad, the seam between the two triangles appears to have some overlap, which causes the fragment shader to be run twice on those pixels. This is a major problem, because it means that I can not assume the shader will be run once for each pixel, and I can’t assume that only one shader instance will be accessing a pixel’s data structures at a time.

I’m using what I think is a pretty standard technique for drawing the quad: currently, I’m calling glDrawElements(GL_TRIANGLES) on two triangles defined in clip space.

I pass in the following vertices to my vertex buffer:
{
{1.0f, 1.0f, 0.5f, 1.0f},
{-1.0f, 1.0f, 0.5f, 1.0f},
{1.0f, -1.0f, 0.5f, 1.0f},
{-1.0f, -1.0f, 0.5f, 1.0f}
}

And my index buffer contains:
{0, 1, 2, 1, 2, 3}

And here’s my vertex shader:

#version 420

layout(location=0) in vec4 in_position;
layout(location=1) in vec2 in_tex;

smooth out vec2 texture_coords;

void main()
{
gl_Position = in_position;
texture_coords = in_tex;
}

How do I know the shader is being run twice on the seam? I wrote a test shader that atomically increments a pixel in an image corresponding to the fragment’s pixel coordinates. The diagonal of the image has all '2’s while everywhere else is 1. When I change the order of the vertices, the '2’s appear along the other diagonal.

I’ve tested this on a radeon HD 7970 and HD 5450 on Ubuntu 11.04 and a Geforce 670 GTX on Mint 13, and it shows up in all of those cases. Curiously, when I try it on windows, the problem does not appear.

I’m wondering if anyone has run into a similar problem before, has advice on how to define the triangles such that the seams are removed, or a different method of rendering the fullscreen quad.

I have no idea how this could happen but why don’t you render a full screen triangle instead of a full screen quad? Just render a triangle large enough that it covers all pixels. This way you definitely cannot experience any seams.

I just tried that. I end up with 1 or 2 lines along which the fragment shader appears to run twice. I’m guessing that what’s going on is the fixed function pipeline clips the triangle and converts it into several smaller ones that fill up the screen, but there still seems to be some overlap.

ortho projection ? Perspective projection is the only thing I can think of that would cause this.

If you have MSAA enabled, you might want to look into centroid sampling for texture coordinates.

I noticed the same thing in Windows but it was a couple of releases of driver ago. I haven’t checked with the most recent drivers

No projection at all; I’m passing the vertices in NDC. Take a look at my vertex shader.

I’m not using MSAA currently.

That’s good to hear. Maybe this will be fixed in the future.

Thanks for the suggestions, guys.

Could you test with this in the fragment shader:


layout(early_fragment_tests) in;

[QUOTE=Ilian Dinev;1241246]Could you test with this in the fragment shader:


layout(early_fragment_tests) in;

[/QUOTE]

No apparent effect.

I tried compiling with clang instead of gcc - still the same. So I’m pretty confused as to what’s going on. It seems strange that both NVidia and ATI drivers would have the same problem specifically on linux.

How exactly are you writing to the other image? If it’s not through the blend unit, it might be incrementing whenever the fragment program is run, not whenever it actually produced a pixel. If so, that would explain the doubling up, as on most chips fragment programs get run in 2x2 blocks – thus running the fragment program twice on every pixel along the diagonal.

The clipping shouldn’t be generating new triangles as long as your fullscreen tri stays within a reasonable range. IIRC, for +/- 1 NDC space, the vertices you need are (-1, -1) (3, -1) (-1, 3). http://www.gamedev.net/topic/568915-fullscreen-triangle/ has a bit more detail on it.

when I try to render a fullscreen quad, the seam between the two triangles appears to have some overlap, which causes the fragment shader to be run twice on those pixels. This is a major problem, because it means that I can not assume the shader will be run once for each pixel, and I can’t assume that only one shader instance will be accessing a pixel’s data structures at a time.

That should not be possible. All writes from any fragment shaders that execute outside of a triangle’s boundaries should be discarded. What is the fragment shader you use to detect this?

If MSAA is enabled, you would get two fragments with each having one half of the samples masked. You could enable per-sample shading plus atomic counters to see if MSAA is active.

Yes, but that’s both normal and to be expected.

Is the overlapping region a single pixel wide, or is it sometimes a couple pixels wide and blocky? If a single pixel, that sounds like a precision issue. If more than one pixel and blocky, it’s likely that you’re seeing an artifact from the GPU invoking your fragment shader in 2x2 (or larger) groups of pixels for efficiency. Normally this is not visible because the extra fragments are discarded, but if you make accesses to main memory then there can be side effects.

Either way, the easiest solution may be to change how you render your full screen quad. Try using a single triangle that is larger than your viewport and fully contains it. This is typically more efficient anyway as it avoids the wasted fragments along the seam between two triangles due to the pixel grouping mentioned above.

Edit: I take back the bit about accessing main memory from a helper fragment having side effects. Looks like that is not supposed to happen, though there could be a bug you a hitting if so. From http://www.opengl.org/registry/specs/ARB/shader_image_load_store.txt

(22) If implementations run fragment shaders for fragments that aren’t covered by the primitive or fail early depth tests (e.g., “helper pixels”), how does that interact with stores and atomics? RESOLVED: Stores will have no effect. Atomics will also not update memory. The values returned by atomics are undefined.

Does having them wound the same direction have any effect?

eg. Instead of:
{0, 1, 2, 1, 2, 3}
using
{0, 1, 2, 1, 3, 2}.

AFAICT if either way is used only one polygon should produce a fragment along the common edge, so you’ll probably need to wait for a driver fix. Triangles wound the same direction may be a more tested path though.

Have you tried this with drawing your fullscreen quad as a single triangle strip using glDrawArrays?

[QUOTE=AlexN;1241304]
Either way, the easiest solution may be to change how you render your full screen quad. Try using a single triangle that is larger than your viewport and fully contains it. This is typically more efficient anyway as it avoids the wasted fragments along the seam between two triangles due to the pixel grouping mentioned above.[/QUOTE]

This was suggested in the second post of the thread, and he tried it in the third.

Sigh.

Triangle strip, not triangle.

As well as potentially providing a solution, a strip (not a single triangle) can provide useful diagnostic info for the OP too, such as confirmation of the MSAA theory mentioned above.

All of this is coming from the perspective of trying to help the OP rather than score points off other posters.

I missed a word; sorry about that.

Though I don’t see how it matters if it’s a strip or two triangles. I seriously doubt the rasterizer would treat it any differently. It’s not like it even sees anything but individual triangles. Multisampling would work the same way as well, regardless of whether it’s a strip or individual triangles.

Indeed, I would be very concerned about the validity of the rendering pipeline if turning it into a strip had any effect at all.