shadowmap gets extruded at the borders -black bars

[ehmdjii]

hello,

i am using old-school shadowmapping in my game and i have the following problem:

the shadowmap doesn't cover all of the visible frustum, so objects in the far distance dont have a shadow. thats ok!

but objects that are on the border of the shadowmap, make the outmost pixel of the shadowmap black. and this leads to these pixels being extruded to infinity. this means i have black bars going away from objects that are partially in the shadowmap.

i hope you can understand my problem, if not i can post a screenshot.

what can i do about this issue?
i have already tried all different clamping-modes.

thanks!

[Nighthawk]

Modulate the resulting brightness with a spotlight texture in lightspace(same space as the shadowmap).

So everything fades to dark at the border, looks better(like a torchlight spot).

[ehmdjii]

thank you! but actually i would like the opposite effect. everything outside of the shadowmap should not be shadowed but lit, since this is for an outdoort-scene.

[Nighthawk]

You could use the same technique, but modulate the shadow darkening factor with the spotlight brightness(assuming you are using a fragment shader).

So the shadows are dark in the center and the shadowing effect gets lighter towards the border.

Might look better with a very soft spotlight border.

[despoke]

I used to fix this issue by setting a texture border colour to the shadow map texture (either 0.0 or 1.0 depending on if you want the outside area to be in shadow or in the light).

However, isn't this feature (texture border colour) being deprecated with openGL3?

[arekkusu]

Yes, texture borders are deprecated. So now instead of letting the hardware automatically clamp to your color, you get to add a couple branches per fragment in your shader.

[ehmdjii]

thank you, so i was wondering how to use texture-borders correctly? the spec says:

"The value of width is 2m (or 2m+2 if there's a border), where m is a nonnegative integer."

so that means if my texture size originally is 512 it should then be 514? right?

unfortunately this doesn't seem to work and the shadowmap looks like it has a staircase-pattern.

[Nighthawk]

In theory, you should be able to generate your own border if the texture borders don't work.

After each shadowmap pass, draw 4 lines in (almost) far plane distance into the shadow map top/bottom/left/right texels. With depth test off and depth write on.

Then sample the shadowmap with normal clamp. Never tried that though.

[ehmdjii]

thank you! that is a good idea!

infact i used glScissor to lock the border. it works now! thanks!

[arekkusu]

Note, CLAMP_TO_BORDER and texture borders are two different things. They are both deprecated in GL3.

CLAMP_TO_BORDER with a constant border color is immensely useful for exactly your problem with shadows, or similar situations like antialiasing a primitive with a texture. All current hardware supports this, and it is still in DX10. I don't understand why GL3 deprecates it.

Texture borders are a different story. A lot of hardware doesn't support border texels (although recent nvidia hardware does) and they are generally a giant pain for driver writers. I entirely understand why they are deprecated.