PDA

View Full Version : preventing self-shadowing



Vexator
12-05-2006, 09:57 AM
i'm feeling a bit stupid.. i have z-fail shadow volumes which are working fine aside from the well-known glitches with self-shadowing on spheres etc. so i'd like to add an option which disables self-shadowing. i thought i could do so simply by rendering the extruded silhouettes as usual and then render the mesh itself in the stencil buffer.. but with which stencil op/func do i have to render the mesh?!


glStencilFunc( GL_ALWAYS, 0, ~0 );

// render silhouette pass 1
glCullFace( GL_FRONT );
glStencilOp( GL_KEEP, GL_INCR, GL_KEEP );
pkGroup->RenderSilhouette();

// render silhouette pass 2
glCullFace( GL_BACK );
glStencilOp( GL_KEEP, GL_DECR, GL_KEEP );
pkGroup->RenderSilhouette();

// which stencil op/func changes do i need here?!

// render mesh itself
pkGroup->Render();thanks in advance :)

Humus
12-05-2006, 11:10 AM
GL_ZERO ought to do the trick, or GL_REPLACE with stencilRef as zero.

Vexator
12-05-2006, 11:39 AM
thanks!
like this?


glStencilOp( GL_KEEP, GL_ZERO, GL_KEEP );
pkGroup->Render();at first it seemed to work fine, but look what happens if i move the character below the floor:

http://www.vexator.net/neu-1.jpg

i have to find a way to make it only do this if the mesh is not occluded i guess?

Komat
12-05-2006, 02:24 PM
One way that might work with one limitation:

Render shadow volumes for everything except for objects that should be not self-shadowed. Render the objects that should be not self-shadowed with lighting from that light. Without clearing the stencil render shadow volumes for the objects that were skipped in first step. Render rest of the lit geometry.

Obviously the problem is that objects that are not self-shadowed will not cast shadows on other such objects. I think that this is what Doom 3 does, if I correctly understand talk (http://www.gamedev.net/community/forums/topic.asp?topic_id=266373) made by Carmack on one Quakecon.

Humus
12-06-2006, 11:24 AM
Originally posted by Vexator:
like this?No. Like this:

glStencilOp( GL_KEEP, GL_KEEP, GL_ZERO );

Vexator
12-07-2006, 08:10 AM
this does not do anything :p thanks, though!

Humus
12-08-2006, 06:23 AM
Could you list your rendering passes and the stencil calls for each of them?

Vexator
12-08-2006, 08:53 AM
void World::RenderLighting()
{
// get active camera
Camera *pkCamera = Camera::GetActiveCamera();

// render one pass for each light source
for( unsigned int a = 0; a < m_vpkVisibleLightsSet.size(); a++ )
{
//get light
Light *pkLight = m_vpkVisibleLightsSet[a];

// enable stencil test
glEnable( GL_STENCIL_TEST );
Core::Get()->GetRenderer()->ClearFramebuffer( Renderer::BUFFER_STENCIL );

if( pkLight->GetCastShadows() )
{
// setup states
glStencilFunc( GL_ALWAYS, 0, ~0 );
glColorMask( false, false, false, false );

// for all models..
for( unsigned int i = 0; i < pkLight->m_vpkShadowCastingGroupsSet.size(); i++ )
{
// activate extrusion shader
Core::Get()->GetRenderer()->ActivateShader( m_pkExtrusionShader );

// get current group
Group *pkGroup = pkLight->m_vpkShadowCastingGroupsSet[i];

// get light to model gl_Vertex
Vector3D kLightVector = pkLight->GetTranslation()-pkGroup->GetParent()->GetTranslation();

// re-compute silhouette
pkGroup->ComputeSilhouette( kLightVector );

// z-fail algorithm
glCullFace( GL_FRONT );
glStencilOp( GL_KEEP, GL_INCR, GL_KEEP );
pkGroup->RenderSilhouette();

glCullFace( GL_BACK );
glStencilOp( GL_KEEP, GL_DECR, GL_KEEP );
pkGroup->RenderSilhouette();

// ------- HACK: try to prevent self-shadowing -------

glStencilOp( GL_KEEP, GL_KEEP, GL_ZERO );

// push matrix
glPushMatrix();

// apply model's view matrix
Core::Get()->GetRenderer()->UploadMatrix( pkGroup->GetParent()->GetViewMatrix(), false );

// render group (unshaded)
pkGroup->Render( 0 );

// Pop matrix
glPopMatrix();

// -----------------------------------------------------------
}

// deactivate shaders
Core::Get()->GetRenderer()->DeactivateShaders();

// reset states
glStencilFunc( GL_EQUAL, 0, ~0) ;
glStencilOp( GL_KEEP, GL_KEEP, GL_KEEP );
glColorMask( true, true, true, true );
}

// enable blending
glEnable( GL_BLEND );
glBlendFunc( GL_ONE, GL_ONE );

for( unsigned int i = 0; i < pkLight->m_vpkIlluminatedGroupsSet.size(); i++ )
{
// get current group..
Group *pkGroup = pkLight->m_vpkIlluminatedGroupsSet[i];

// push matrix
glPushMatrix();

// apply model's view matrix
Core::Get()->GetRenderer()->UploadMatrix( pkGroup->GetParent()->GetViewMatrix(), false );

// check for custom shader, otherwise use to default lighting shader
Shader *pkShader = pkGroup->GetCustomShader() ? pkGroup->GetCustomShader() : m_pkLightingShader;

// activate shader
Core::Get()->GetRenderer()->ActivateShader( pkShader );

// render it
pkGroup->Render( pkShader );

// deactivate shaders
Core::Get()->GetRenderer()->DeactivateShaders();

// Pop matrix
glPopMatrix();
}

// disable blending
glDisable( GL_BLEND );

// disable stencil test
glDisable( GL_STENCIL_TEST );
}
}

Komat
12-08-2006, 09:10 AM
The anti self-shadowing render needs to be done using glDepthFunc( GL_LEQUAL ). Otherwise the depth test fails because the object geometry is already contained within the depth buffer with the same depth values.

One additional note. The glStencilOp( GL_KEEP, GL_KEEP, GL_ZERO ) render will erase all shadows on corresponding object that are casted by objects whose shadow volumes were rendered before so you might wish to introduce some ordering into the rendering of shadow volumes.

Vexator
12-08-2006, 09:47 AM
The anti self-shadowing render needs to be done using glDepthFunc( GL_LEQUAL )why have i been switching the depth function at all? Oo removed those switched and set GL_LEQUAL in the ctr, still does not help.. thanks anyway :D

Komat
12-08-2006, 10:11 AM
Originally posted by Vexator:
why have i been switching the depth function at all? Oo removed those switched and set GL_LEQUAL in the ctr, still does not help.. thanks anyway :D What is the result after the change? Is the self-shadowing visible, it is the same as on the image you post or something different is wrong?

Komat
12-08-2006, 10:21 AM
You can use the GLIntercept (http://glintercept.nutty.org/) to see changes made to depth and stencil buffers by individual OGL calls. This might give you a clue what does not work as expected.

Vexator
12-09-2006, 12:55 AM
Is the self-shadowing visible, it is the same as on the image you post or something different is wrong?the self-shadowing is still visible.