Geometry shaders and depth FBOs

Hi

Are there any special rules regarding rendering to an FBO when there’s a geometry shader involved?

Context:
I’m trying to implement shadow mapping in a scene which includes different types of objects being rendered in different ways:

Meshes, rendered in the standard way through VBOs and shaders
Point sprite particles, rendered with a vertex and fragment shader
Quad particles, sent in as points but expanded to quads in a geometry shader

I’m doing shadow mapping in the standard way, which involves creating a depth FBO and rendering to it the object’s depths from the light’s point of view. The meshes and point sprites work fine, but the depths of the geometry shader quad particles aren’t being rendered to the FBO.

Here’s the standard (non-FBO) depth buffer. Note the large circular-shaped particles in the foreground.

Here’s the same view rendered to the FBO. Note that only the depths of the meshes and point sprite particles are drawn.

My FBO is set up as follows:

const bool Renderer::generateShadowMap(FBO & f, string & error) {
	uint depthTexID;
	uint width = f.getWidth();
	uint height = f.getHeight();
	glGenTextures(1, &depthTexID);
	glBindTexture(GL_TEXTURE_2D, depthTexID);
	glTexImage2D(	GL_TEXTURE_2D,		0,					GL_DEPTH_COMPONENT,
					width,				height,				0,
					GL_DEPTH_COMPONENT, GL_FLOAT,	NULL);

	float border[] = {1.0, 0.0, 0.0, 0.0};

	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_BORDER);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_BORDER);

	glTexParameterfv(GL_TEXTURE_2D, GL_TEXTURE_BORDER_COLOR, border);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_COMPARE_REF_TO_TEXTURE);
	glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FUNC, GL_LESS);

	glActiveTexture(GL_TEXTURE1);
	glBindTexture(GL_TEXTURE_2D, depthTexID);
	
	uint fID;
	
	glGenFramebuffers(1, &fID);
	glBindFramebuffer(GL_FRAMEBUFFER, fID);

	glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_TEXTURE_2D, depthTexID, 0);
	glDrawBuffer(GL_NONE);

	if(!checkFBOstatus(error)) {
		cerr << error << endl;
		return false;
	}
	glClearDepth(1.0);
	glClear(GL_DEPTH_BUFFER_BIT);
	glBindFramebuffer(GL_FRAMEBUFFER, 0);
	glActiveTexture(GL_TEXTURE0);
	f.setFBOID(fID);
	f.setTexID(depthTexID);

	return true;
}

The shaders that should render the depths of the quad particles are:


//VERTEX SHADER
#version 400

in vec3 aPosition;
in float aSize;

out float gSize;

uniform mat4 vWorldView;
void main() {
	gSize = aSize;
	vec4 v4Vertex = vec4(aPosition.xyz, 1.0);

	vec4 transformedVertex = vWorldView * v4Vertex;
	
	gl_Position = transformedVertex;
}



//GEOMETRY SHADER
#version 400

layout (points) in;
layout (triangle_strip, max_vertices = 4) out;

in float gSize[1];
uniform mat4 vgProjection;

void main() {
	vec4 oPosition; 
	
	float halfSize = gSize[0] * 0.5;

	oPosition = (vec4(-halfSize, -halfSize, 0.0, 0.0) + gl_in[0].gl_Position);
	gl_Position = vgProjection * oPosition;
	EmitVertex();


	oPosition = (vec4(halfSize, -halfSize, 0.0, 0.0) + gl_in[0].gl_Position);
	gl_Position = vgProjection * oPosition;
	EmitVertex();


	oPosition = (vec4(-halfSize, halfSize, 0.0, 0.0) + gl_in[0].gl_Position);
	gl_Position = vgProjection * oPosition;
	EmitVertex();


	oPosition = (vec4(halfSize, halfSize, 0.0, 0.0) + gl_in[0].gl_Position);
	gl_Position = vgProjection * oPosition;
	EmitVertex();

	EndPrimitive();
}



//FRAGMENT SHADER - empty as depth should be written automatically
#version 400

void main() {
}

Depth testing is enabled, of course.

I’ve run this through gDebugger which can’t detect any errors at all. I’ve tried telling the fragment shader to actually draw things but that doesn’t make a difference either.

The only difference between the point sprite particles and the quad particles is the use of the geometry shader. I’m rendering both at the same time and with the same uniform values (and I’ve run gDebugger to make sure).

This occurs on an nVidia GTX 560 Ti with driver version 8.7.13.142 and an ATI Mobility Radeon HD 5650 (not sure about the driver version, but updated within the last few months, I think).

Have I made any obvious mistakes or are there certain rules I need to know about?

Thanks

I managed to fix the problem, but I have another issue.

The solution: the shadows weren’t being drawn as I had enabled front face culling for the rendering of shadows. Changing the order of the vertices in the geometry shader fixed it.

New issue:
Because the depth of the shadow is essentially the same as the depth of the particle itself, every particle will be affected by its own shadow.

Image (ignore the “particle transparency” bit - it’s part of the HUD)

I’ve tried applying a polygon offset through glPolygonOffset (for lines, points and fill) but it only seems to affect the shadows cast by the mesh objects (not built in a geometry shader). I’ve also tried messing about with the gl_FragDepth (so the particle fragments would be treated as closer to the camera, and the particle shadows would be treated as being further away), but that gave no change other than the particle or its shadow completely disappearing when the depth was outside of the 0 to 1 range.
I also tried changing the screen-space depth of the shadow vertices themselves. Lower offsets removed the self shadow but also displaced the positions of the shadows on other objects.

Image

I then tried to draw the particles closer to the screen so that their screen-space depths would be lower than that of the shadows. Applying the offset before the projection transformation (but after applying the modelview matrix) displaced the particles according to the camera position but otherwise did not affect the shadows. Applying it after projection seemed to change the distance at which the particle could be seen but didn’t change the shadow at all.

What am I doing wrong? Is this issue actually possible to solve?

Thanks