Hello,
I have a depth-texture-only FBO and a main scene FBO of same size. The depth-only FBO is single sampled, the main scene one has MSAA.
- The depth-only FBO is cleared with glClearDepth(0.0);
- Front faces of a cube are rendered to the depth-only FBO using glCullFace(GL_BACK)
- Back faces of this cube are rendered to the MSAA FBO using glCullFace(GL_FRONT) with same MVP-Matrix etc (everything is exactly the same)
When doing 3), in the fragment shader, the depth value of the corresponding front face is queried like this
float Z = texelFetch(TextureFrontFaces, ivec2(gl_FragCoord.xy), 0).x;
Everything I want to do with that works as expected BUT I noticed that for fragments at at the silhouette of the cube (i.e. where back and front faces are touching)
the Z value is the ClearDepth 0.0 from step 1) and not the one from a front face.
I wonder how this is possible? I was reading how MSAA is implemented and basically how MSAA works but still don’t get the reason.
I mean, even though step 3) will populate for a pixel multiple depth samples covered by the fragment I would
assume for every fragment the shader should find a front face depth value in the front face texture.
When I disable MSAA (i.e. by setting a single sampled main scene FBO) or when I disable MSAA rasterization
via glDisable(GL_MULTISAMPLE) then the fragment shader will never lookup a ClearDepth 0.0 (as expected)!
So it MUST be the MSAA that causes this strange behavior.
Anyone knows why this can happen? To me it seems as if a fragment is generated in step 3)
which was not generated in step 2), but only when MSAA is turned on for the main FBO.
Help is really appreciated!
Note: The fact that glDisable(GL_MULTISAMPLE) at least seams to avoid this on my graphics card/driver setup,
might indicate that the issue has something to do with the coverage mask used during MSAA?!
The spec says the effect of glEnable(GL_MULTISAMPLE) is
When multisampling is enabled (see glEnable with argument GL_MULTISAMPLE) a “fragment mask” is computed for each fragment
generated by a primitive. This mask reflects the amount of the pixel covered by the fragment, and determines the frame buffer
samples that may be affected by the fragment.
Without multi-sampling or anti-aliasing (GL_POLYGON_SMOOTH), rasterising a triangle affects exactly those fragments whose centre lies within the triangle. For a fragment whose centre lies exactly on an edge, the determination as to whether the fragment is inside or outside is unpredictable, but it’s consistent in the sense that if the edge is shared by two triangles, the fragment will be inside exactly one of the triangles.
With multi-sampling or anti-aliasing, rasterisation affects any fragment which intersects the triangle, regardless of whether its centre is inside or outside.
Thank you so much! Then of course it can happen that a fragment generated during MSAA rasterization does not find a corresponding front-face fragment which was generated without MSAA.
What I still don’t get is why glDisable(GL_MULTISAMPLE) seems to solve the problem? Reading the linked ARB spec above I don’t really
see that turning this off leads to fragments being ignored as without MSAA when the centre is not part of the triangle. But maybe I
missed something?
This makes me also unsure if it solves the problem on any graphics card/driver implementation.
If you disable multi-sampling, the behaviour is the same for both front and back faces: fragments are only generated if their centres are inside the triangle.
Note that for any closed mesh, any point (including but not limited to fragment centres) which lies inside the silhouette lies inside an even number of triangles (half of which are front-facing, half back-facing). For a convex closed mesh, there will be exactly one front-facing and one back-facing triangle containing that point.
The behaviour of rasterisation when multi-sampling is disabled probably isn’t mentioned in the ARB_multisample specification; it’s described in the core OpenGL specification (e.g. §14.6.1 in the 4.5 specification, which also describes the behaviour with anti-aliasing in §14.6.3 and multi-sampling in §14.6.6).
Ok thanks, then it is probably safe to just call glDisable(GL_MULTISAMPLE) 
Btw this is what the spec and ARB txt says:
If MULTISAMPLE is disabled, multisample rasterization of all primitives is
equivalent to single-sample (fragment-center) rasterization, except that the frag-
ment coverage value is set to full coverage. The color and depth values and the
sets of texture coordinates may all be set to the values that would have been as-
signed by single-sample rasterization, or they may be assigned as described below
for multisample rasterization.