Fragment program to mimic rectangular clipping plane

I wrote this fragment program to highlight part of a 3D model that is in SectPlane plane and falls completely into the triangles (in this plane) defined as (P1, P2, P3) and (P4, P2, P3)

varying vec4 Position;  // position of the fragment (initialized as Position = gl_Vertex; in vertex program)
uniform vec4 SectPlane; // equation of the plane (in the same coord. system as Position)
uniform vec3 P1;        // four points in this plane to form two triangles (P1, P2, P3) and (P4, P2, P3)
uniform vec3 P2;        
uniform vec3 P3; 
uniform vec3 P4;
bool SameSide( vec3 p1, vec3 p2, vec3 a, vec3 )
{
    vec3 cp1 = cross( b-a, p1-a );
    vec3 cp2 = cross( b-a, p2-a );
    return dot(cp1, cp2) >= 0;
}
bool PointInTriangle( vec3 p, vec3 a, vec3 b, vec3 c )
{
   return SameSide(p, a, b, c) && SameSide(p, b, a, c) && SameSide(p, c, a, b);
}
void main()
{
   float Threshold = 0.01 + 10.0 * gl_FragCoord.z - 1.0;
   float Dist = abs( dot( Position, SectPlane ) );
   if ( Dist > Threshold ) { discard; }
   if ( !( PointInTriangle( vec3( Position ), P1, P2, P3 ) || PointInTriangle( vec3( Position ), P4, P2, P3 ) ) ) { discard; }
   gl_FragColor = vec4( 1.0, 0.0, 0.0, 0.0 );
}

The problem is i have to make the thickness of this section independant from the distance to the camera. I tried to use Threshold = 0.01 + 10.0 * gl_FragCoord.z - 1.0; as a criterium, but this brings no success at all.

Please, any suggestions and improvements are welcome!

Instead of using gl_FragCoord.z you could compute distance in vertex shader, and pass it as varying variable in fragment shader.

Have just added this computation to a vertex shader:

Depth    = length( vec3(gl_ModelViewMatrix * gl_Vertex) );

and changed fragment shading code to:

Threshold = 0.01 + pow( 0.0005 * (Depth - 200.0), 2 );

This also dosn’t seem to behave the way i want it to. I just want this section to behave like a line, i.e. be always some pixels in width regardless of the distance to the camera.

Seems like you misunderstood me a bit. Try this:

Threshold = 0.01 * Depth;

I’ve tried it, also 0.01 was too big, i used 0.0003. But it looks even worse than my formula with squared depth. Thickness of the section zooms in and out with the model. I’m upset

Then it must be something else. Look at the first line in your fragment shader:

varying vec4 Position;  // position of the fragment (initialized as Position = gl_Vertex; in vertex program)

Shouldn’t position be transformed by modelview matrix?
Make sure you have all values in proper coordinate spaces.

Position and SectPlane are in the same coordinate space, so the calculation of dot product Position*SectPlane gives me the correct distance of the point from this plane. After that i compare it to Threshold and discard the fragment if it is outside of the tolerance. I guess i compute the Threshold totally wrong but i don’t have any idea how it could be fixed.

Position*SectPlane gives me the correct distance
Try mapping abs(dot(Position,SectPlane)) to color and see if you get what’s expected. You never know for sure until you see it with your own eyes.

I guess i compute the Threshold totally wrong
My bet is still on coordinate spaces. Threshold is just too simple to compute - it’s just distance to vertex rescaled by some constant. Not much can go wrong here (unless you have some fancy transformations in projection matrix, too).

If you won’t be able to solve this then I’d like to know what your vertex shader looks like. Also, fragments of code where you set shader uniforms and matrices would be helpful.

I’m out of the office, so code fragments will be only 'morrow in the morning. I swear this dot product is correct since the section itself is rendered exactly as it was expected, so distance from a plane is computed the right way, so is PointInTriangle() - i could see it with my own eyes . Just the width goes wrong.

Calculating the radial distance from camera point in vertex shader and sending it to fragment shader will not work for two reasons:

1. The perspective division, which is the thing you need to compensate if the tickness should remain the same, operates on distance from eye plane.
2. The interpolation of radial distance will offten result in values that do not correspond to correct radial distance inside the triangle.

When the perspective division is done, the vector is divided by the W coordinate. To calculate the correct threshold for some distance, you should imho multiply the base treshold value with W corresponding to the distance.

So, here they are:

vertex program:

varying vec4 Position;
varying float Depth;
void main()
{
   Position = gl_Vertex;
   Depth    = length( vec3(gl_ModelViewMatrix * gl_Vertex) );
   gl_Position = ftransform();
}

and recent fragment program:

varying vec4 Position;  // position of the fragment (initialized as Position = gl_Vertex; in vertex program)
uniform vec4 SectPlane; // equation of the plane (in the same coord. system as Position)
uniform vec3 P1;        // four points in this plane to form two triangles (P1, P2, P3) and (P4, P2, P3)
uniform vec3 P2;        
uniform vec3 P3; 
uniform vec3 P4;
bool SameSide( vec3 p1, vec3 p2, vec3 a, vec3 ){    
   vec3 cp1 = cross( b-a, p1-a );
   vec3 cp2 = cross( b-a, p2-a );
   return dot(cp1, cp2) >= 0;
}
bool PointInTriangle( vec3 p, vec3 a, vec3 b, vec3 c )
{   
   return SameSide(p, a, b, c) && SameSide(p, b, a, c) && SameSide(p, c, a, b);
}
void main()
{   
   float Threshold = 0.0003 * Depth;   
   float Dist = abs( dot( Position, SectPlane ) );   
   if ( Dist > Threshold ) { discard; }   
   if ( !( PointInTriangle( vec3( Position ), P1, P2, P3 ) || PointInTriangle( vec3( Position ), P4, P2, P3 ) ) ) { discard; }   
   gl_FragColor = vec4( 1.0, 0.0, 0.0, 0.0 );}

I guess instead of length() i should use .z coordinate, right?

How could i compensate the perspective division? Should i use gl_FragCoord.w? This doesn’t seem to work either.

I guess instead of length() i should use .z coordinate, right?

The .w coordinate.

How could i compensate the perspective division? Should i use gl_FragCoord.w? This doesn’t seem to work either.
By multiplying the treshold with .w coordinate. Did you tried multiplying by the 1/gl_FragCoord.w ?

I’ve tried .z, .w, 1/,w and also .z*.w and .z / .w - result is bad and if i use .w the thickness depends on size of the viewport.

if i use .w the thickness depends on size of the viewport
You have to scale it by expected section width and viewport size. This will work for aspect ratios close to 1:1 since viewport can have different width and height, but I suggest you first make it work with 1:1.

I’m stuck with it. It doesn’t want to behave correctly. The size of the section is changing if i zoom my model in and out.

Can you post some screenshots?

Ohh… only on monday. I’m out of the office already. I will post a couple of screenshots asap.

Take your time, Sergey K. Don’t strain yourself.

Perhaps this weekend you could stop by the library and check out some math books.

Here are the screenshots:

You could see how red section changes it’s width. I just don’t want this to happen.

Which treshold equation did you use for the screenshots? The zooming is done by moving the object or by changing the field of view of the camera?

Added: If you are zooming by changing field of view, you need to multiply the treshold by appropriate constant. For example, if you are using something like:

glFrustum(left * zoom, right * zoom, bottom * zoom, top * zoom, znear, zfar)

then the treshold should be something like:

good_looking_constant * w_coordinate * zoom

The optical thickness of the line will still change in dependency on the mesh geometry and orientation of the plane.