I have a texture with transparent parts on which I’m doing specular lighting using the separate specular calculation in the fixed function pipeline. The areas of the texture with 0 alpha are still getting specular highlights, but I don’t want them to. Preferably, I’d like to modify the specular highlight by the texture’s alpha.
Is it possible to do this in fixed function OpenGL?
I can’t go to shaders because this is already a fallback path for people who don’t have shaders. I’ve looked through the various env extensions, but I couldn’t figure out a way to do this (the specular lighting contribution gets stuck in the secondary colour (right?) and there’s no way to access the secondary colour in one of the env extensions, yes?).
Nvidia’s register combiners extension has access to the secondary colour, so I could do this on GF1 - GF4 cards, but what to do for ATI cards? I guess one option is not using the separate specular lighting model and then the specular lighting contribution would become part of the primary colour (yes?) and be modulated correctly by alpha when you set the correct glTexEnv() environmnet.
Any ideas?
edit - I just realized that this would work if I could use a subtractive (a*fg + (1-a)*bg) blend mode, but unfortunately, I have to work in an additive blend mode (fg + (1-a)*bg), because its a lot more common used than subtractive blending in our industry.