Lighting normals/color ignored with GL_REPLACE.

[JCipriani2]

If I draw a textured quad, with normals, and I have these states set:

glEnable(GL_LIGHTING);
glEnable(GL_LIGHT0);
glEnable(GL_COLOR_MATERIAL);
glTexEnvi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_REPLACE);

The textures are always rendered full bright and lighting is not applied. It is important in this application that I use GL_REPLACE rather than setting glColor4f(1, 1, 1, 1). Why isn't the lighting being applied?

Thanks,
Jason

[Bob]

GL_REPLACE means that the output of the texture combiner is the the texture color. You need GL_MODULATE to multiply the primary color with the texture.

[JCipriani2]

Thanks for your reply.

I think I want to use GL_REPLACE and not GL_MODULATE, though. I do not want to multiply the primary color with the texture, but I do want the lighting calculations to be done.

The reason why is that I have a large array of data that I use with glInterleavedArrays(GL_T2F_C4F_N3F_V3F), and I can switch between using either the vertex colors, OR a texture. It's not convenient in this application for me to modify the color data to (1,1,1,1) in the array, or to create a separate data array one for the textured object and another for the vertex colored object. So I am using GL_REPLACE so that the color data in the array is ignored when I'm drawing the object with a texture on it. When I do this, though, shading from the lights also seems to stop working.

I might just not know how it works -- when lighting calculations are done (given normals) is the resulting color part of the "primary color" (rather than some separate multiplication step later on that also applies to textures)? If that's the case is my only option to use GL_MODULATE with the color set to white?

Thanks,
Jason

[-NiCo-]

There are more options.

1 - Don't use glInterleavedArrays as the draw call:

The effect of
 
      InterleavedArrays(format, stride, pointer);
 
is the same as the effect of the command sequence
 
      if (f ormat or stride is invalid)
         generate appropriate error
      else {
         int str;
         set et , ec , en , st , sc , sv , tc , pc , pn , pv , and s as a function
            of table 2.5 and the value of f ormat.
         str = stride;
         if (str is zero)
            str = s;
         DisableClientState(EDGE FLAG ARRAY);
         DisableClientState(INDEX ARRAY);
         DisableClientState(SECONDARY COLOR ARRAY);
         DisableClientState(FOG COORD ARRAY);
         if (et ) {
            EnableClientState(TEXTURE COORD ARRAY);
            TexCoordPointer(st , FLOAT, str, pointer);
         } else
            DisableClientState(TEXTURE COORD ARRAY);
         if (ec ) {
            EnableClientState(COLOR ARRAY);
            ColorPointer(sc , tc , str, pointer + pc );
         } else
            DisableClientState(COLOR ARRAY);
         if (en ) {
            EnableClientState(NORMAL ARRAY);
            NormalPointer(FLOAT, str, pointer + pn );
         } else
         DisableClientState(NORMAL ARRAY);
         EnableClientState(VERTEX ARRAY);
         VertexPointer(sv , FLOAT, str, pointer + pv );
}

See OpenGL spec for details on the values of the pointers and strides. You can set ec to false if you don't want to draw the primary colors.

2 - Use shaders

[Bob]

I might just not know how it works -- when lighting calculations are done (given normals) is the resulting color part of the "primary color" (rather than some separate multiplication step later on that also applies to textures)? If that's the case is my only option to use GL_MODULATE with the color set to white?

The lighting equation completely overrides the primary color, yes. That means the only way your own primary color can have any effect is if you enable color material, which you have done according to your first post. And if this is undesired, then you have it enabled without wanting it.

The color material feature is used to have some material property track the user primary color. This can be done because the primary color is overridden by the lighting equation, and can now be used for something else instead. This is what happens here; your material changes with the color in the array.

While there are some solutions, like already mentioned, it sounds more like a design problem which should be dealt with from the start instead. That is, remove the source of the problem instead of fixing the code around it. If the user primary color affect the lighting in an undesired way, then instead of setting the parameters to go around the problem, you disable what makes the user primary color affect the lighting in the first place.

[JCipriani2]

Thanks for your answers. For now I've gone with something like #1 in -NiCo-'s reply. As for fragment shaders... I'm working on that ;-) .

Jason