i am using old-school shadowmapping in my game and i have the following problem:
the shadowmap doesn’t cover all of the visible frustum, so objects in the far distance dont have a shadow. thats ok!
but objects that are on the border of the shadowmap, make the outmost pixel of the shadowmap black. and this leads to these pixels being extruded to infinity. this means i have black bars going away from objects that are partially in the shadowmap.
i hope you can understand my problem, if not i can post a screenshot.
what can i do about this issue?
i have already tried all different clamping-modes.
thank you! but actually i would like the opposite effect. everything outside of the shadowmap should not be shadowed but lit, since this is for an outdoort-scene.
I used to fix this issue by setting a texture border colour to the shadow map texture (either 0.0 or 1.0 depending on if you want the outside area to be in shadow or in the light).
However, isn’t this feature (texture border colour) being deprecated with openGL3?
Yes, texture borders are deprecated. So now instead of letting the hardware automatically clamp to your color, you get to add a couple branches per fragment in your shader.
In theory, you should be able to generate your own border if the texture borders don’t work.
After each shadowmap pass, draw 4 lines in (almost) far plane distance into the shadow map top/bottom/left/right texels. With depth test off and depth write on.
Then sample the shadowmap with normal clamp. Never tried that though.
Note, CLAMP_TO_BORDER and texture borders are two different things. They are both deprecated in GL3.
CLAMP_TO_BORDER with a constant border color is immensely useful for exactly your problem with shadows, or similar situations like antialiasing a primitive with a texture. All current hardware supports this, and it is still in DX10. I don’t understand why GL3 deprecates it.
Texture borders are a different story. A lot of hardware doesn’t support border texels (although recent nvidia hardware does) and they are generally a giant pain for driver writers. I entirely understand why they are deprecated.