I want to do normal blending with OpenGL, like the blending between layers in Photoshop for example, but it doesn’t seem possible with glBlendFunc.
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) does not work as expected because if I start with a transparent buffer, render a solid primitive and then render a semi-transparent, the result is semi-transparent even if I did render something totally solid in between.
That’s just all wrong, and it does not work for me.
How can I do normal blending where the alpha value is calculated as 1 - (1-As)*(1-Ad) ?
I haven’t found a general solution to this problem which I consider to be a gaping flaw in the OpenGL standard, but I have found a workaround for my case, so I thought I would let anyone who are interested know …
I use glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
First I render a solid background, that covers most of the buffer, the edges are still transparent.
Then I render the semi-transparent primitives twice, first with the previously mentions blending and then with glBlendFunc(GL_ONE, GL_ONE), texture turned off and glColor4f(0.0, 0.0, 0.0, 1.0);
The solution does not seem very elegant and it seems suboptimal, but it works for me. However, this only works because all the semi-transparent primitives are rendered entirely inside the bounds of the solid background.
I fail to see how you managed to not make it work :
// init part
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glEnable(GL_BLEND);
// in main loop
glClearColor (0.0f, 0.0f, 0.0f, 0.0f);
glClear (GL_COLOR_BUFFER_BIT);
glPushMatrix ();
glBegin (GL_TRIANGLES);
glColor4f (1.0f, 1.0f, 0.5f, 1.0f);
glVertex2f (0.0f, 1.0f);
glVertex2f (0.87f, -0.5f);
glVertex2f (-0.87f, -0.5f);
glColor4f (0.0f, 1.0f, 0.5f, 0.5f);
glVertex2f (0.8f, 1.0f);
glVertex2f (0.0f, 0.0f);
glVertex2f (1.0f, 0.0f);
glEnd ();
glPopMatrix ();
Works perfectly for me.
Beware of the ‘OpenGL is not working’ … it can be right on super fancy new stuff with beta drivers or for features almost never used. Not on something like that.
Edit: I suppose you are using textures. best way to control transparency like in photoshop is to use :
// allow to easily change color with a single glColor call
glEnable(GL_COLOR_MATERIAL);
// multiply texture with base color
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE);
/////////// for each layer :
// change transparency, t in [O,1] range
glColor3f(1.0f,1.0f,1.0f,t);
//bind texture
//draw stuff
///////
Your code works in the same way that I explained.
------------ Screenshot -----------------
I know it won’t work because the blend function is by definition not usable for what I’m trying to do.
SRC_ALPHA, ONE_MINUS_SOURCE_ALPHA calculates the resulting alpha as AsSa+AdDa which in this case is, 0.50.5+1.0(1-0.5) = 0.75
I need the resulting alpha to be 1.0, like in photoshop blending, or window manager blending, but there is no suitable blending function for this. 
If I understand well, you want to separate rgb and alpha blending, there are extension to adress this (if supported by your hardware) : GL_EXT_blend_func_separate (and maybe GL_EXT_blend_equation_separate).
http://oss.sgi.com/projects/ogl-sample/registry/
By the way, how did you made the text appear behind… AH ! I see, when you say a transparent buffer, you mean some kind of overlaying GL drawing on top of desktop image. Am I right ?
Yes, I use OpenGL laid directly over the desktop, or directly in a transparent window, to be exact.
I’m not sure if I want to separate color and alpha blending, I’ll take a look at the link. I do want to use as few extensions as possible for maximum compatibility, though.
I think I’ll just stick to the work around that I found, since it seems to do the job well, in my case.
Edit: I looked at GL_EXT_blend_func_separate and with that I would almost be able to do what I want:
BlendFuncSeparateEXT(SRC_ALPHA, ONE_MINUS_SRC_ALPHA, SRC_ALPHA, ONE);
would have been nice. The alphas would have been additive instead of multiplicative which would be ideal, but I could have lived with that.
Thanks for helping out.