FBO with Color0 + Depth attachment = wrong shadows

I’m experiencing some issues trying to render my scene and display it on a fullscreen quad.

I first render depth to a shadowMap for all shadow-casters in my scene. I then render the scene where all shadow-receivers get the shadowMap in the fragment shader.

This works just fine when I render directly to the backbuffer, but if I try to render to an FBO color0 attachment, and then send that texture to a fullscreen quad for final backbuffer rendering, the shadows gets all wrong.

I would think it’s got to do with how I set up my FBO with depth and color texture attachements. If someone could give me some help on thisone I’d be very happy!

some code:

void GenericManager::initFBO()
{
	glGenFramebuffers(1, &fbo);
}

void GenericManager::initShadows()
{	
	glGenTextures(1, &depthShadowMap);
	glBindTexture(GL_TEXTURE_2D, depthShadowMap);
	{
		// GL_LINEAR does not make sense for depth texture. However, next tutorial shows usage of GL_LINEAR and PCF
		glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
		glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);

		// Remove artefact on the edges of the shadowmap
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP);
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP);

		depthShadowMapSize = 1024;
		// No need to force GL_DEPTH_COMPONENT24, drivers usually give you the max precision if available 
		glTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, depthShadowMapSize, depthShadowMapSize, 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_BYTE, 0);

	} glBindTexture(GL_TEXTURE_2D, 0); //Done setting up the texture
}

void GenericManager::initFinal()
{	
	glGenTextures(1, &finalMap);
	glBindTexture(GL_TEXTURE_2D, finalMap);
	{
		// GL_LINEAR does not make sense for depth texture. However, next tutorial shows usage of GL_LINEAR and PCF
		glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
		glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);

		// Remove artefact on the edges of the shadowmap
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
		glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

		finalMapSize = 1024;
		// No need to force GL_DEPTH_COMPONENT24, drivers usually give you the max precision if available 
		glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB, finalMapSize, finalMapSize, 0, GL_RGB, GL_UNSIGNED_BYTE, 0);

	} glBindTexture(GL_TEXTURE_2D, 0); //Done setting up the texture

	//Create a the final fullscreen quad to render the scene's final framebuffer texture to
	finalNode = factory->create(Scene::NODE_PLANE);
	finalNode->setShader(cl_format("%1/%2", basePath, "Resources/Shaders/final"));
	finalNode->GetProperty<siut::simd::Vec3f>("Size") = siut::simd::Vec3f(10.0f, 10.0f, 0.0f);
	finalNode->setTexId8(finalMap);
	finalNode->add(finalCamera);
	finalCamera->translate(0.0f, 0.0f, 13.0f);
	finalCamera->lookAt(finalCamera->getGlobalPos(), finalNode->getGlobalPos(), siut::simd::Vec3f(0.0f, 1.0f, 0.0f));
}

void GenericManager::postInitFBO()
{
	glBindFramebuffer(GL_FRAMEBUFFER_EXT, fbo);
	{
		GLenum fboStatus;
		// Instruct openGL that we won't bind a color texture with the currently binded FBO
		
		glDrawBuffer(GL_NONE);
		glReadBuffer(GL_NONE);

		// attach the texture to FBO depth attachment point
		glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_TEXTURE_2D, depthShadowMap, 0);

		glDrawBuffer(GL_COLOR_ATTACHMENT0);
		glReadBuffer(GL_NONE);

		// attach the texture to FBO color0 attachment point
		glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, finalMap, 0);

		// check FBO status
		fboStatus = glCheckFramebufferStatus(GL_FRAMEBUFFER);
		if(fboStatus != GL_FRAMEBUFFER_COMPLETE)
			printf("GL_FRAMEBUFFER_COMPLETE failed, CANNOT use FBO
");

	} glBindFramebuffer(GL_FRAMEBUFFER, 0); // switch back to window-system-provided framebuffer
}

I call these at initialization of my scene manager:

	lightCamera = static_cast<Core::PerspectiveCamera*>(factory->create(Scene::NODE_PERSPECTIVE_CAMERA));
	lightCamera->perspective(40.0f, 1.0f, 1.0f, 10000.0f);

	finalCamera = static_cast<Core::PerspectiveCamera*>(factory->create(Scene::NODE_PERSPECTIVE_CAMERA));
	finalCamera->perspective(40.0f, 1.0f, 1.0f, 10000.0f);

	initFBO();
	{
		initShadows();
		initFinal();
	} postInitFBO();

The next two snippets show how I render the scene using the FBO

void GenericManager::display()
{
	siut::simd::Mat4f lightView;
	siut::simd::Vec3f lightDir;

	glBindFramebuffer(GL_FRAMEBUFFER, fbo);	//Rendering offscreen
	{
		//////////////////////////////
		// PASS 1
		// DRAW DEPTH FOR SHADOW
		//////////////////////////////
		glDrawBuffer(GL_NONE);
		glReadBuffer(GL_NONE);
);
		glClear(GL_DEPTH_BUFFER_BIT);
		glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE); //Disable color rendering, we only want to write to the Z-Buffer

		// Cull switching, rendering only backface, this is done to avoid self-shadowing
		glCullFace(GL_FRONT);
		if(lightCamera)
		{
			siut::simd::Vec3f lightPos = siut::simd::Vec3f(580.0f, 280.0f, 680.0f);
			siut::simd::Vec3f lightTarget = siut::simd::Vec3f(64.0f*8.0f, 100.0f, 64.0f*8.0f);
			lightDir = lightPos - lightTarget;
			lightCamera->lookAt(lightPos,
								lightTarget,
								siut::simd::Vec3f(0.0f, 1.0f, 0.0f));

			lightView = lightCamera->getViewMatrix();
			coreManager->setActiveCamera(lightCamera);
		}
		displayScene(false, true, lightView, lightDir);

		//////////////////////////////
		// PASS 2
		// DRAW COLOR ATTACHMENT
		//////////////////////////////
		glDrawBuffer(GL_COLOR_ATTACHMENT0);
		glReadBuffer(GL_COLOR_ATTACHMENT0);

		glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE); 
		glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

		glCullFace(GL_BACK);
		if(coreManager->getStateManager()->getCurrentState()->getPerspectiveCamera())
			coreManager->setActiveCamera(coreManager->getStateManager()->getCurrentState()->getPerspectiveCamera());
		displayScene(true, false, lightView, lightDir);

	} glBindFramebuffer(GL_FRAMEBUFFER, 0);

	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
	displayFinal();
}

void GenericManager::displayFinal()
{
	IRenderInfo *info = finalNode->getRenderInfo();
	if(info == NULL || finalNode->isHidden())
		return;

	coreManager->setActiveCamera(finalCamera);
	renderer->displayInfo(info, finalNode, 0, siut::simd::identityMatrixf(), siut::simd::Vec3f());
}

This results in the following render:

If I change my display function, so that it becomes this instead:

void GenericManager::display()
{
	siut::simd::Mat4f lightView;
	siut::simd::Vec3f lightDir;

	glBindFramebuffer(GL_FRAMEBUFFER, fbo);	//Rendering offscreen
	{
		//////////////////////////////
		// PASS 1
		// DRAW DEPTH FOR SHADOW
		//////////////////////////////
		glDrawBuffer(GL_NONE);
		glReadBuffer(GL_NONE);

		glClear(GL_DEPTH_BUFFER_BIT);
		glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE); //Disable color rendering, we only want to write to the Z-Buffer

		// Cull switching, rendering only backface, this is done to avoid self-shadowing
		glCullFace(GL_FRONT);
		if(lightCamera)
		{
			siut::simd::Vec3f lightPos = siut::simd::Vec3f(580.0f, 280.0f, 680.0f);
			siut::simd::Vec3f lightTarget = siut::simd::Vec3f(64.0f*8.0f, 100.0f, 64.0f*8.0f);
			lightDir = lightPos - lightTarget;
			lightCamera->lookAt(lightPos,
								lightTarget,
								siut::simd::Vec3f(0.0f, 1.0f, 0.0f));

			lightView = lightCamera->getViewMatrix();
			coreManager->setActiveCamera(lightCamera);
		}
		displayScene(false, true, lightView, lightDir);

	} glBindFramebuffer(GL_FRAMEBUFFER, 0);

	glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE); 
	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

	glCullFace(GL_BACK);
	if(coreManager->getStateManager()->getCurrentState()->getPerspectiveCamera())
		coreManager->setActiveCamera(coreManager->getStateManager()->getCurrentState()->getPerspectiveCamera());
	displayScene(true, false, lightView, lightDir);
}

then it renders the shadows just fine:

Anyone spot some immediate mistakes? I’m not sure, but I have a feeling that renderbuffers is something I might have to use to get this right… any advice that can make me smarter would be highly appreciated though!

This section looks mighty suspect, because here you’re clearing out the depth buffer in “fbo” you just generated in Pass 1.

I’m not sure, but I have a feeling that renderbuffers is something I might have to use to get this right…

No, texture or renderbuffer either one should be just fine. And you’re gonna need a texture because you can only bind textures (not renderbuffers) a shader sampler to feed into your scene rendering pass.

Hm, ok. So then there’s something here I don’t fully grasp. I thought that my first pass would write to the depth texture, and the second pass would write to the finalMap texture. In the FBO initialization I tried to bind the depthMap to glDrawBuffer(GL_NONE) and the finalMap to glDrawBuffer(GL_COLOR_ATTACHMENT0).

I’ve worked some more on this, and seperating the two textures into each their own FBO fixes the problem, but is not free from problems of their own When I seperate into two FBOs, the water plane suddenly starts to hover over the terrain and follow the camera movement for no appearant reason I suspect that I have other issues, and that seperating into two FBOs resolved the issue I had problems with here.

Sounds fine.

In the FBO initialization I tried to bind the depthMap to glDrawBuffer(GL_NONE) and the finalMap to glDrawBuffer(GL_COLOR_ATTACHMENT0).

This doesn’t make any sense. glDrawBuffer and glReadBuffer set to which COLOR buffer attachment you are going to draw to or read back from. If you set it to NONE, there is no COLOR buffer. This is a fine thing to do when you are rendering the depth map (DEPTH only, no COLOR). But you are not “binding depthMap to DrawBuffer NONE”. You are just “unbinding any existing color buffer”.

My problem is that, in between your two glClears (both of which clear the DEPTH buffer), there’s nothing I see which unbinds the depthMap from the depth buffer and binds another depth texture or renderbuffer in its place for rendering. So it appears that your 2nd glClear will clear depthMap, which I don’t think is what you want.

Ah, I think I see. Hmm, I’ll have to give it another shot then! Thank you for your input!