ID buffer in geometry shader

[Ares Lagae]

Hi all,

We are experiencing a nasty problem involving accessing an ID buffer in a geometry shader. We have been spending days on this, but did not advance. We appreciate any hints / ideas / ...

We are working on a point sprite renderer.
- The geometry shader generates points on-the-fly on the surface of the incoming triangles.
- In order to avoid generating sprites for invisible geometry we generate an ID buffer first.
- The ID buffer is generated by rendering triangles and writing the primitive ID to an integer texture using an FBO.
- Before emitting a generated point in the geometry shader, we do a texture lookup in the ID buffer to check if the point corresponds to the visible triangle.
- The setup basically works, but we get flicker artefacts.
- Strangely, when doing a similar test in the fragment program, the program setup works perfectly, without flicker artefacts.
- However, this is also much slower, since all sprites are rasterized. We want to prevent rasterization by not generating the points in the first place.

Since the tests work in the fragment shader, I assume the ID buffer itself is correct. But then I really cannot see what can be going wrong.

In the geometry shader, we sample the texture as follows:
- the vertex program computes the clip coordinates

gl_Position = gl_ModelViewProjectionMatrix * gl_Vertex;

- the geometry shader converts these clip coordinates to normalized device coordinates by performing the perspective division, and then to window coordinates by

vec4 position_in[3];
position_in[0] = gl_PositionIn[0] / gl_PositionIn[0].w;
position_in[1] = gl_PositionIn[1] / gl_PositionIn[1].w;
position_in[2] = gl_PositionIn[2] / gl_PositionIn[2].w;
...
// position is a point on the triangle formed by the vertices of position_in
...
ivec2 P = ivec2(((position * 0.5) + 0.5) * textureSize(texture, 0))
...
textureFetch( ... P ...)
...

When doing multiple texture fetches at neighboring positions (+1 and -1) the problem improves but does not go away.

I know this is not an easy problem to solve remotely, but we appreciate any hints, ideas, ...

Best regards,

[Ares Lagae]

I have found the problem.

There was no programming bug.

The ID's of rasterized triangles do not always agree with the ID's of the generated points, because of the discretization.

For example, when generating a point close to the edge of a triangle, it is analytically inside the triangle, but a discretized ID buffer might disagree.

...

Are there robust solutions for this?