What I mean by alpha is opacity. And instead of having just one value (currently called alpha), opacity should have 3 components: one for scaling red, one for green and one for blue. Opacity erpresents how the surface filters color seen through it.
For now, when using (GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA), if the source color is (SCr, SCg, SCb), the source alpha is Sa, the destination color is (DCr, DCg, DCb) and the destination alpha is Da, we have:
final color = (SCr * Sa + DCr * (1 - Sa), SCg * Sa + DCg * (1 - Sa), SCb * Sa + DCb * (1 - Sa))
final alpha = Sa * Sa + Da * (1 - Sa)
Now, with an RGB source opacity (SOr, SOg, SOb) and the blending equation (GL_ONE, GL_ONE_MINUS_SRC_ALPHA) (which could be called (GL_ONE, GL_ONE_MINUS_SRC_OPACITY)), we would have:
final color = (SCr + DCr * (1 - SOr), SCg + DCg * (1 - SOg), SCb + DCb * (1 - SOb))
and, if opacity information in needed in the final framebuffer image (with (DOr, DOg, DOb) being the destination opacity):
final opacity = (SOr + DOr * (1 - SOr), SOg + DOg * (1 - SOg), SOb + DOb * (1 - SOb))
It is easy to see that if the same behavior than the first computation is needed, (SCr, SCg, SCgb) should just be replaced by (SCr * Sa, SCg * Sa, SCb * Sa) (which can be computed in the fragment shader), (SOr, SOg, SOb) by (Sa, Sa, Sa) and (DOr, DOg, DOb) by (Da, Da, Da).
If a monochrome alpha is needed in the destination image (e.g. for generating a PNG with an alpha channel), it can easily be computed like this (with (Or, Og, Ob) being the final opacity computed above):
final alpha = 0.3 * Or + 0.59 * Og + 0.11 * Ob
which is the formula to compute luminance from RGB.
Actually, what we currently call alpha is just a particular case where Or = Og = Ob. Using (GL_ONE, GL_ONE_MINUS_SRC_ALPHA) makes the source color independent of the opacity (but it can be taken into account if we want it) and having an RGB opacity allows each color component to be scaled differently. It just leaves us the choice to do whatever we want, with one easy general case.
Some examples:
Green glass with an HDR envmap for reflection:
vec3 envmapColor; // color read in the HDR envmap
vec3 reflectionColor; // RGB reflection ratio of our material (just like specular color)
gl_FragColor = envmapColor * reflectionColor;
gl_FragOpacity = vec3(1, 0, 1); // green filter
Classical non-premultiplied RGBA texture:
vec3 materialColor; // color of material with lighting computed
vec4 textureColor;
gl_FragColor = materialColor * textureColor.rgb * textureColor.a;
gl_FragOpacity = vec3(textureColor.a);