Cutout blending with minimum passes

Hi People.Haven’t visited for long time here ))

I asked the question also here: https://computergraphics.stackexchange.com/questions/6295/openglcutout-effect-with-hardware-blending-modes-and-alpha

Here is what I want to do. I need to create screen space mask for some rendered geometry,which is generated out of another geometry rendering results.

The easiest way I see (and it works):

  1. Render mask geometry into FBO A.

  2. Render masked geometry into FBO B.

  3. Screen space pass, shader program with input samplers of mask texture from FBO A,and masked texture from FBO B.

  4. Draw screen quad, use alpha inside the fragment shader of texture A ,to mask texture B.

All right? That’s the easiest but also the slowest approach. Because in my case, every object first is rendering into MSAA FBO to fix AA.
So I have to render 2 times,then blit twice,and then render one more time in screen space to apply actual masking.

So I went to this page: Anders Riggelsen - Visual glBlendFunc and glBlendEquation Tool

played around with blending functions,and came to this:

glBlendFuncSeparate(GL_ONE, GL_ZERO, GL_ONE, GL_ONE);
glBlendEquationSeparate(GL_FUNC_ADD, GL_FUNC_SUBTRACT);

My attemp of doing it with hardware blending is described here: https://computergraphics.stackexchange.com/questions/6295/openglcutout-effect-with-hardware-blending-modes-and-alpha

And it doesn’t work. I wonder how can make this effect with less passes.

What’s wrong with just drawing the “shape” image without blending, then everything after that with:


glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_DST_ALPHA);

?

Note that the FBO will need an alpha channel.

You mean rendering the mask normally,and the masked objects with the mode you provided? Than how about masked objects which have alpha transparency? They must be rendered with alpha blending mode on.

In that case, you’re probably going to need to use a shader with both the image and mask textures as sources. There isn’t a blending mode which will give you src_alpha*(1-dst_alpha). If the mask is a hard mask rather than a linear factor, you could use stencilling instead.

As for the question as to why that blending function doesn’t “work”: you’re using a source factor of one and a destination factor of zero, i.e. source colour replaces destination colour, with neither source nor destination alpha having any role (the source and destination alpha factors are both one, so alpha will be accumulated; but alpha isn’t actually used, so it doesn’t really matter what the destination alpha ends up as).

[LEFT]Not exactly true. See ,I use GL_FUNC_SUBTRACT equation for alpha component. So that’s the actual formula for alpha :
(sA1) - (dA1) In my case src texture has alpha = 1 always .But the mask,which is dest texture,has background alpha zero and the circle shape = 1. so the src texture should get zero alpha where dest texture has alpha = 1 and get alpha =1 where dest texture has alpha = 0. Totally makes sense and it looks all right on that website with blending functions.[/LEFT]