Texture Coordinate Sanity Check...

[bobvodka]

I was working on some GLSL based shadow mapping code and for some reason my final image had a dirty great brown section down one side near the light.

I tried for ages to make it go away and finally someone did by doing the perspecitve divide and checking the coords were within [0,1].

Now, while this work and the brown mark does indeed go away it confused me, as the textures in question (shadow map and a texture to represent a spot light, white in the middle fading to black on the outside) has GL_CLAMP_TO_EDGE set and REPEAT as none, yet somehow it seems to end up sampling wrong and thus making this brown mark show up.

So, I wanted todo a sanity check, if I have the above modes set on a texture if I try to sample outside [0,1] it will indeed clamp it inside [0,1] and I don't have todo the work myself in the fragment shader, its definatly the domain of the texture fetching hardware...?

Its probably my code wrong somewhere, but ya know how it is, its nice to have a sanity check when you've been pluggin away at something...

For the record the two fragment shaders in question are;
broken;

 uniform sampler2DShadow shadowMap;
uniform sampler2D colorMap;
 
varying vec4 projCoord;
 
void main()
{
	vec4 shadowValue;
	vec4 lightColor;
	// Check for reverse projection and remove it
	if(projCoord.q < 0.0)
	{
		shadowValue = vec4(0.0);
		lightColor = vec4(0.0);
	}	
	else if(shadowCoord.x >= 0.0 &amp;&amp; shadowCoord.y >= 0.0 &amp;&amp; shadowCoord.x <= 1.0 &amp;&amp; shadowCoord.y <= 1.0)
	{
		shadowValue = shadow2DProj(shadowMap, projCoord);
		lightColor = texture2DProj(projectedLight, projCoord);
	}	
 
	gl_FragColor = shadowValue*lightColor;
 
}

working;

 uniform sampler2DShadow shadowMap;
uniform sampler2D colorMap;
 
varying vec4 projCoord;
 
void main()
{
	vec4 shadowValue = vec4(0.0);
	vec4 lightColor = vec4(0.0);
	vec3 shadowCoord = projCoord.xyz/projCoord.q;
 
 
	if (shadowCoord.x >= 0.0 &amp;&amp; shadowCoord.y >= 0.0 &amp;&amp; shadowCoord.x <= 1.0 &amp;&amp; shadowCoord.y <= 1.0 )
	{
		shadowValue = shadow2D(shadowMap, shadowCoord.xyz);
		lightColor = texture2DProj(colorMap, projCoord);
	}
 
 
	gl_FragColor = shadowValue*lightColor;
 
}

Thanks

[Komat]

The GL_CLAMP_TO_EDGE should work.

In your broken shader the shadowValue and lightColor is not initialized for points before the light which have shadowCoord outside the <0,1> range.

[bobvodka]

right, I thought so...

and it turns out I copied and pasted the code wrong;

 uniform sampler2DShadow shadowMap;
uniform sampler2D colorMap;
varying vec4 projCoord;
void main(){
	vec4 shadowValue;
	vec4 lightColor;
	// Check for reverse projection and remove it
	if(projCoord.q < 0.0)
	{
		shadowValue = vec4(0.0);
		lightColor = vec4(0.0);	}		else 
	{
		shadowValue = shadow2DProj(shadowMap, projCoord);
		lightColor = texture2DProj(projectedLight, projCoord);
	}
 
	gl_FragColor = shadowValue*lightColor;
}

the second 'if' shouldn't have been there *slaps head*

ta for the reply

[Komat]

Do you use mipmaps on the projected texture? If mipmaps are used, the projection will cause selection of smaler mipmap levels when projCoord.q is close to zero and point is far from the light axis.

[bobvodka]

yes, that was it, thanks, I've learnt something new today

Although, I have to admit I'm slightly confused as the mipmap'd texture is only on colorMap, I wouldn't have thought that would have effected things from the point of view of multiplying it with the depth texture, which doesn't use mipmaps.. or am I missing something fundimental here?

[Komat]

With shadowmap that has boundary that always results in full shadow (e.g. minimal possible distance) mipmapping of the colorMap should cause no problem because that boundary will ensure that everything outside the light cone is black.

Without the boundary, the task to remove the light from outside of the cone is left to the colorMap only and selection of lower mipmap will in that case cause the problem.