Per Fragment depth mask

[kRogue]

There are some pretty nifty uses for the following:

Per Fragment depth mask, i.e. a fragment shader can dictate if the depth buffer is to be written with the depth value of the fragment. Interaction with glDepthMask is that it is an AND, i.e. the depth buffer is written to if and only if both glDepthMask is GL_TRUE and the fragment shader does not say "don't write depth". To be precise, introduce a "function" in the fragment shader, "gl_DontWriteDepth()" then if called within the fragment shader prevents the depth buffer from being updated. Another approach is to create a new GLSL built in fragment shader only variable:

bool gl_FragmentDepthMask;

that is initialized as true and if the value of it is false when main() of the fragment shader exits, then the depth value for the fragment is not updated.


Along similar lines, same jazz for stencil buffer would be nice too. [For non-integer color buffers, one can emulate this via blending, for per-channel masking one can still do this with blending via GL_ARB_blend_func_extended].

[Ilian Dinev]

What are the nifty uses, that require this to save you from doing a 2-drawcall solution of alphatest/discard on pass 1 + depthmask(false) on pass 2 ? I mean, it's a thing that has been possible at a tiny extra cost for the past 10-15 years, and no-one came-up with a use for it?

[aqnuep]

I'd be also interested in the use cases. It sounds simple enough that even if current hardware is incapable of doing it, maybe future hardware can easily add support for it.
Please share your use cases, seriously interested!

[kRogue]

The two pass solution is what I do now, but if the fragment shader is heavy, it is not a happy place.

Roughly speaking the idea is a follows: when one runs the fragment shader, one can compute a coverage value. Now, if that coverage is not "enough", then one would like to draw that fragment blended with the background [I am not talking about transparent stuff at all]. That bites as then that requires rendering back to front.. so, break it into two separate buffers:

1. initialization: setup 2 target FBO with RGBA(opaque color), RGBA(transparent)
2. set opaque target to blending add with operation being GL_SRC_ALPHA and GL_ONE_MINUS_SRC_ALPHA
3. set transparent target with blending _OFF_

Then the shader code is like this

out vec4 opaque_color_target;
out vec4 transparent_color_target;
 
if(coverage<SOME_THRESHOLD)
{
   opaque_color_target=vec4(0.0, 0.0, 0.0, 0.0);
   gl_DepthFragmentMask=false;
 
   transparent_color_target=vec4(color, coverage);
}
else
{
  /*
    Note that we overwrite the transparent completely to nothing
   */
   opaque_color_target=vec4(color, 1.0);
   gl_DepthFragmentMask=true;
 
   transparent_color_target=vec4(0.0, 0.0, 0.0, 0.0);
}

Once the buffers are drawn then one presents the buffers, essentially

RGB=opaque_color_target.rgb + transparent_color_target.a*transparent_color_target.rgb

With this thing we get coverage based anti-aliasing where edges do not overlap (edges overlapping is really not that common enough in terms of number of pixels for me to freak). My target is not handling "jaggies" of primitives, but rather expensive font shaders that draw glyphs as quads and I want to render opaque UI's in an out-of-order fashion.

Currently, I emulate this with a two pass technique, but that means the font shader (or another expensive shader) gets run twice which, well, sucks. Also there is no guarantee that the front most "AA-crud-edge" will gets its day in the sun. On the subject of AA-based on primitives, there is this: http://www.khronos.org/registry/gles...age_sample.txt which kind of smells like the above, but it only handles the stuff at a primitive level, not a fragment level which is needed for font rendering and for that matter any fragment shader that uses discard.

One can naturally extend this to the idea of multiple edges on a fragment supported in much the same way one does out of order transparency (which in fact this reduces to)... but the main use case is to just avoid running a primitive and all its fragment stuff again to get around something that might be just an API oversight.

[aqnuep]

Why don't you simply use discard and image store on the transparent color target in the following way?

out vec4 opaque_color_target;
uniform image2D transparent_color_target;
 
if (coverage < SOME_THRESHOLD)
{
   imageStore(transparent_color_target, ivec2(gl_FragCoord.xy), vec4(color, coverage));
   discard;
}
else
{
   imageStore(transparent_color_target, ivec2(gl_FragCoord.xy), vec4(0.0, 0.0, 0.0, 0.0));
   opaque_color_target = vec4(color, 1.0);
}

This should be equivalent to your proposed approach and should have the same performance.

[kRogue]

I had always figured that imageStore was slower than usual drawing.. also imageStore is like using a 1000000 pound hammer when I really do not need something so... powerful.... also I need to be a touch paranoid about the sync issues... I admit that my memory is a haze of how imageStore behaves on repeatedly setting the same texel, i.e. the last fragment logically may not be the last fragment that bops the imageStore.. but I cannot remember well enough all the sync issues of it (usually folks use imageStore in combination with atomic ops so that one usually only writes to one location at most once in a frame), indeed from the spec:

* The relative order of invocations of the same shader type are undefined. A store issued by a shader when working on primitive B might complete prior to a store for primitive A, even if primitive A is specified prior to primitive B. This applies even to fragment shaders; while fragment shader outputs are written to the framebuffer in primitive order, stores executed by fragment shader invocations are not.

this makes me a touch nervous, since I can imagine that if multiple primitives are being processed, and if they overlap, then who/what gets the last word on store could get icked up potentially...