GLSL + Clipping

I already asked that question in another thread of mine, but the topic was a bit misleading. So let me ask again:

Is it possible to write a shader that returns a “boolean” array, whether a triangle is visible or not after clipping?

For example you got 3 triangles
#1 gets clipped once
#2 doesnt get clipped
#3 gets fully clipped and discarded

the result would be something like that (in pixel colors) WHITE-WHITE-BLACK

I already managed to produce a shader that tells me if a edgepoint of that triangle is visible.

vertex shader:
void main()
{
gl_Position = gl_Color;
gl_ClipVertex = ftransform();
}

fragment shader:
void main() {
gl_FragColor = vec4(1.0,1.0,1.0,1.0);
}

gl_Color is used to position the information in a array and everything is rendered to a pbo. That works if I set glPolygonMode to GL_POINT, but it only recognizes the edges! I need something equivalent to that, but it needs to check every point of the triangle. Simply setting the glPolygonMode just gives me no results (why!?).

Regards
shiBBy2k9

There are different ways to clip a vertex: simple scanline conversion, and analytical clipping in the [-1;1] clipspace. I have experience only with the scanline version, which can’t be of use to you.

The gl_ClipVertex that you try is for additional clip-planes. The gpu’s internal clipping will do its job only if you’re really drawing triangles, but in your task you have to draw points.

So, research analytical clipping of a triangle in normalized device coordinates.

So I have to calculate that manually? Would it be better to do that on cpu?

Depends on:

  • is the code too branchy?
  • is the count of triangles small?
  • do you need the results immediately (it can take whole 1-2ms to sync the cpu with the gpu).

Btw, an idea on how to check triangle-frustum intersection quickly (but with false positives) (fortunately no false negatives):
the triangle in clipspace… get its AABB. Intersect the AABB with the [-1;1] clipspace AABB.