I have some suggestions to improve the Z-buffer:
1) Store second nearest depth . Proposed by Woo/Wang/Molnar like 20 years ago and used in the Pixar's Renderman 11 (midpoint shadow filtering). Very useful for shadows, transparency, translucency and subsurface scattering. I bet will be very easy to implement in HW and could be very nice.
(find "midpoint", "zero bias shadow")
(looks like Carmack wants to use this for QuakeWars)
Anybody know how can be done this with the current SM3.0 hardware? I tryed all but the floating point epsilon errors prevent me to do it well in the 2nd fragment shader pass...
2) Irregular Z-buffer. Instead of storing all the pixels equally-spaced in the depth buffer, allow to store it in a variable distance way.
Could be used too in shadows.
3) 32bits stencil. 8bits stencil can be not enough. Using 32bits ( with can yield us a 64bits D32S32 depth buffer) you could be pixel IDs for a G-buffer or whatever controlling their values on Zfail/Zpass.
4) Integration of the ZBuffer with the future "blend shader". If a future blend shader is comming, please allow us to read/write the Zbuffer. Kill the "can't read ZBuffer because the FBO is in use" thing ( I know, I know... performance problems... but some things like per-pixel sorting in one pass or the mentioned 2nd depth requires read AND write access... )
5) Depth cubemaps. I know, they are coming... But I want them with FETCH NxN feature. Basically like the ATI's fetch4 but greater. I could want to get, in one textureCube call, the 5x5 block of depths surrounding an specific pixel ( for example, to perform a fast PCF ). Ofc I could to this with multiple textureCube calls, but the fetch thing is better because won't have to re-divide by the greater coordinate to get the face because was already calculated... Also to do the shadow comparison by HW in an NVIDIA way.
6) Dynamic-in shader depth query. Strange concept this... Imagine I want to see if a pixel is in a shadow or not... Basically I want to execute an occlusion query for the current pixel viewed by a specific light position. So, inside the framebuffer fragment shader, it uses an FBO of 1x1 pixel. Then puts the camera in the light and project the pixel to camera space. Then creates a 1D-line zbuffer with all the in-line-of-sigth triangles ( using a modified guard band clipping to know the triangles hit by the ray )... Basically does a IsPointVisibleFrom(3DWorldPoint,3DWorldOtherPoint) using a 1pixel zbuffer. What I want is an GLSL instruction to launch inside the pixel shader occlusion queries without having to generate a shadow map. I want all in one pass/rendering for speed and not to write the light visibility function in a texture that suffers aliasing.
We could do pseudo-raytracing with this, but only gives us only a boolean visibillity function using oclussion queries. Speed gonna be a problem, but consider we aren't going to call this masively in the fragment shader ( 8/16 times max and only inside the shadow dynamic branch )
I know, sounds scary but can kill definitely the shadow map aliasing and perform amazing pseudoraytraced shadows.