Visual artifact resulting from reading texture coordinate in geom shader

This problem is very strange, but I’m going to take my time and explain it in depth.

I have a shader that takes a series of dashed lines and turns them into “broken” ribbons – i.e. each group of dashed lines is transformed into a ribbon that has gaps (and obviously the gaps are courtesy of the dash lines). The end of each ribbon also has an arrow. Here’s an illustration of what it does:

Before:

After:

The vertex shader is the following (really simple):


varying vec4 texCoordV;
void main(void)
{
  gl_FrontColor  = gl_Color;
  gl_Position = ftransform();
  texCoordV = gl_MultiTexCoord0;
}

now for the geometry shader (slightly more involved):


#version 120
#extension GL_EXT_geometry_shader4 : enable
uniform float scale;
uniform vec3 axis;
// expects a line with two points
varying in vec4 texCoordV[2];

void main()
{
  vec4 offset = gl_ModelViewProjectionMatrix*vec4(axis[0]*scale,
                                                  axis[1]*scale,
                                                  axis[2]*scale, 0.0);
  float arrowScale;
  vec4 texCoord;
  // first edge of ribbon
  for(int i = 0; i < gl_VerticesIn; i++)
  {
    arrowScale = 1.0;
    texCoord = texCoordV[i];
    // arrow head
    if (texCoord.x >= 0.9)
      arrowScale = 1.5*(1.0 - texCoord.x)/0.1;
    gl_FrontColor = gl_FrontColorIn[i];
    gl_Position = gl_PositionIn[i] - offset*arrowScale;
    gl_TexCoord[0] = texCoordV[i];
    EmitVertex();
  }
  // second edge of ribbon
  for(int i = 0; i < gl_VerticesIn; i++)
  {
    arrowScale = 1.0;
    texCoord = texCoordV[i];
    // arrow head
    if (texCoord.x >= 0.9)
      arrowScale = 1.5*(1.0 - texCoord.x)/0.1;
    gl_FrontColor = gl_FrontColorIn[i];
    gl_Position = gl_PositionIn[i] + offset*arrowScale;
    gl_TexCoord[0] = texCoordV[i];
    EmitVertex();
  }
  EndPrimitive();
}

fragment shader:


uniform bool detectFaces;
void main()
{
  vec4 finalColor = gl_Color;
  
  if (detectFaces)
  {
    // (default)
    finalColor = vec4(0.4, 0.4, 0.4, 1.0);
    // front faces are brighter
    if (gl_FrontFacing)
      finalColor = vec4(0.8, 0.8, 0.8, 1.0);
  }
  gl_FragColor = finalColor;
}

Let me explain the implementation:

  1. Take a line strip representing a ribbon segment as input. This line strip has an axis of rotation that is given to the geometry shader. Each segment has a 1D texture coordinate (like a parameter) that tells us how far we are along the ribbon – 0.0 at beginning, 1.0 at end. Since there are gaps, each segment will have its own unique range of texture values – like a subset.

  2. Extend the line strip into a ribbon using the axis of rotation. So if the ribbon is supposed to be 2 inches wide, create an edge 1 inch on one side of the line strip using the axis vector, and create another edge 1 inch on the other side. Connect the edges to get a ribbon.

  3. If the texture coordinate of our geometry shader’s point indicates that we are close to the head (like 0.9 or above), then we are in the arrow head region. The arrow head starts off as slightly larger than the ribbon’s width and linearly comes to a fine point.

There are few other details that may be less relevant: I do outlining during rendering by rendering a thick wireframe of the ribbons and offsetting them into the depth buffer, then I render the opaque surfaces. The fragment shader also shades the front and back faces a certain color – this is only done when I render the surfaces, not the outlines. In the future I will do sophisticated version of silhouetting…

Anyway, for the most part things look OK. I do notice that when I rotate the camera (i.e. quaternion trackball), the geometry can “pop out” and do strange stuff like this:

Notice the giant triangle strip that “sticks out” from these giant ribbons (I had to scale up the ribbons sizes for this screenshot)? That’s the artifact, and it’s hard to capture it because it goes away if I rotate the camera slightly. or if I minimize the window and force a re-draw. So it’s quite annoying and difficult to reproduce. Here’s another, except in this case the opaque surfaces sticks out, not the outline:

Two things that can remove the bug:
–If I ignore the texture coordinate. Even if I set the output of the texture coordinate of the vertex (gl_TexCoord[0] = texCoordV[i]), the bug occurs again. It also happens when I do the if statement (if (texCoord.x >= 0.9)). Doesn’t happen if I comment those two lines and leave this one uncommented: texCoord = texCoordV[i].
–If I draw each ribbon as one complete segment instead of individual segments. I suppose I could break up a complete line segment into a smaller pieces inside of geometry shader instead of feeding the shader a series of broken line segments, but that’s for future work.

Anyway I’m not sure why my current implementation would cause such visual oddities…and not sure why my experimental solutions manage to the fix the problem, even though they do not give me the renderings that I want.

In case you are wondering, I’m running this on a Macbook Pro 8,2 w/a Radeon HD 6750M (default openGL for this platform 2.1 ATI-1.0.29). OS version 10.8.2. Thanks…

Hmm. I guess this is too tough. I will probably just divide up the lines in the geometry shader.