How to use multiple output shaders??

Hi,

I am developing a small OpenGL 3.3 core application and I would like to do some color selection to figure out what has been hit by the mouse. I cannot use ray-tracing for it on my application because I am using some fragment shader tricks to “cut” some of my objects.

I managed to get a shader program with multiple array inputs, uniforms and samplers on both vertex and fragment shaders running and rendering correctly after following a series of tutorials (www.arcsynthesis.org and lighthouse3D). But I still can’t get how to handle multiple outputs on the fragment shader.

On my fragment shader I have the following lines declaring my outputs:
layout(location = 0) out vec4 outputColor;
layout(location = 1) out uvec4 colorPicker;

What I would like to know:

  1. What is the precision of the “colorPicker” output? Is is 32 bits x 4 or some value that is defined by the hw? Ideally I would like to set it to be a single uint_32.

  2. How do I change which one of the outputs is draw to the screen?

  3. How do I read a pixel data from the “Color Picker” output? I have tryed:


    	GLubyte data[4];
    	glReadBuffer(outputColorLocation);
    	glReadPixels(x,y,1,1,GL_RGBA,GL_UNSIGNED_BYTE,&data);
        ...
    	glReadBuffer(colorPickerLocation);
    	glReadPixels(x,y,1,1,GL_RGBA,GL_UNSIGNED_BYTE,&data);
    	...

Where the outputColorLocation/colorPickerLocation is the value returned by glGetFragDataLocation but it does not seem to work.

  1. I have also tried to use glDrawBuffers but I couldn’t understand how is the relation between the buffers and the frag data locations. Is there anywhere I could read more about it?

  2. I would also like to know what happens if I load multiple shaders and do multiple render passes. How the colorPicker data will mix? Is it possible to control it?

  3. I am also looking for a good book that deals with openGL core 3.3+, especially with shaders and how to handle the comunication between the GPU/CPU.

Thanks

What is the precision of the “colorPicker” output?

It’s a vector of 4 unsigned integers. Per the GLSL spec, integer types have 32 bits of precision.

Ideally I would like to set it to be a single uint_32.

Then write a unit instead of a uvec4.

How do I change which one of the outputs is draw to the screen?

Use glDrawBuffers.

How do I read a pixel data from the “Color Picker” output?

You have to use _INTEGER for your pixel transfer format when downloading or uploading integer image data. So your command should be:


glReadPixels(x, y, 1, 1, GL_RGBA_INTEGER, GL_UNSIGNED_BYTE, &data);

Also, glReadBuffer takes a buffer name, not a fragment shader output location.

  1. I have also tried to use glDrawBuffers but I couldn’t understand how is the relation between the buffers and the frag data locations. Is there anywhere I could read more about it?

Yes, there is.

How the colorPicker data will mix? Is it possible to control it?

You cannot blend with integer data. So you’ll overwrite whatever was there.

I am also looking for a good book that deals with openGL core 3.3+, especially with shaders and how to handle the comunication between the GPU/CPU.

Can you be more specific about what you’re looking for?

Okay, I am still not getting it… After reading the material you sent me I tryed to add the following to my render loop:


        GLenum buf[] = {GL_BACK_LEFT, GL_COLOR_ATTACHMENT1  };
        glDrawBuffers(2,buf);

Since outputColor is on 0 and colorPicker is on 1 buf[outputColor]=GL_BACK_LEFT and buf[colorPicker]=GL_COLOR_ATTACHMENT1, correct?

After that I changed my query code to:


GLubyte data[4];
glReadBuffer(GL_FRONT);
glReadPixels(x,y,1,1,GL_RGBA,GL_UNSIGNED_BYTE,&data);
    	...
glReadBuffer(GL_COLOR_ATTACHMENT1);
glReadPixels(x,y,1,1,GL_RGBA_INTEGER,GL_UNSIGNED_BYTE,&data);
    	...

I am getting output like this (looks like output color on both variables):


color = (195,175,148,255)
picker = (195,175,148,255)
color = (172,145,118,255)
picker = (172,145,118,255)
color = (156,139,123,255)
picker = (156,139,123,255)
color = (57,88,109,255)
picker = (57,88,109,255)

I am calling this on my render loop:


 glUniform4ui(pickerIDLocation,0,id,1,1);

Where ID is an uint from 1 to 16 (depending on the object) and pickerIDLocation is the location I get from getUniformLocation.

The relevant part of the fragment shader:



uniform uvec4 pickerID;
layout(location = 0) out vec4 outputColor;
layout(location = 1) out uvec4 colorPicker;

void main()
{
	vec4 imageColor = texture(colorTexture,colorCoord);
	bool outside = isOutside();
	bool online  = isInternalLine();
	bool visible = isVisible();
	
	if (outside)
		discard;
	else if (online) {
		outputColor = imageColor;
		colorPicker = uvec4(pickerID.xyz,255);
	}
	else if (visible) {
		outputColor = imageColor;
		colorPicker = uvec4(pickerID.xyz,255);
	}
	else discard;
}

Do I have to create a FBO object to do this?

[quote]How the colorPicker data will mix? Is it possible to control it?

You cannot blend with integer data. So you’ll overwrite whatever was there.
[/QUOTE]
Ok, but if I understand correctly when I use “discard” on a fragment it will not overwrite what was there.

[quote]I am also looking for a good book that deals with openGL core 3.3+, especially with shaders and how to handle the comunication between the GPU/CPU.

Can you be more specific about what you’re looking for? [/QUOTE]
I already have a solid linear algebra, calculus and geometry knowledge. And I would like to gain a solid background on OpenGL funcionality as well as on 3D graphics altought my imediate needs are 2D rendering (for the application I am writing). I would like to have a text that would explain all the core funcionality : what are the main objects, main functions, how to pass data to/from/between shaders/CPU programs,

There are two types of frame buffers. The default frame buffer, and FBO that you allocate. You can’t mix targets between these. The default framebuffer can only use targets like GL_AUXi, GL_BACK_LEFT, etc, while FBOs that you allocate can use GL_COLOR_ATTACHMENTi. Consider using two different shaders. One for rendering to screen, and one to use for picking?

If you have multiple outputs, you need buffers for each of them. When rendering to the screen, there are pre defined buffers. But if you use FBOs, then you have to allocate the buffers on your own.

Another way to go is to only use one output of the shader, GL_BACK_LEFT. If you are in picking mode, don’t swap the back buffer with the front buffer. But that would be a problem if you can’t get enough picking information from RGBA.

Regarding study material, I think http://www.arcsynthesis.org/gltut/ is quite good? Although it is not complete yet. Problems with tutorials and books are that many of them are still stuckin the immediate mode of thinking, even if they are based on core functionality.

There are two types of frame buffers. The default frame buffer, and FBO that you allocate. You can’t mix targets between these. The default framebuffer can only use targets like GL_AUXi, GL_BACK_LEFT, etc, while FBOs that you allocate can use GL_COLOR_ATTACHMENTi. Consider using two different shaders. One for rendering to screen, and one to use for picking?
[/QUOTE]

Gl3w is not exposing GL_AUX symbols for me… I will try to use the FBO’s and post the result.

I thought about using this solution but them I would have to run the vertex shader two times. It is not a problem for my application but I don’t think it is an elegant solution.

I find arcsynthesis tutorials great but in the current state they stop just before explaining the FBOs :wink:

I find arcsynthesis tutorials great but in the current state they stop just before explaining the FBOs

I wouldn’t say “just before”; I won’t touch FBOs for a good 3-4 more tutorials. I still have to do projective texturing, bump mapping, blending, and probably forward rendering before I get to render to texture.

I am trying to get the FBOs to work. Is this the right way of doing it? Is there a way to attach the default GL_BACK buffer to one of the slots of my FBO?

  1. Create a new FBO
  2. Create a new renderBuffer. Initialize it to the current screen widthxheight.
  3. Create a new texture. Initialize it to the current screen widthxheight.
  4. Bind both to the FBO.
  5. Use drawBuffers to atach the render buffer to colorPicker and the texture to colorOutput.
  6. Draw everything.
  7. Unbind the FBO.
  8. Bind the texture to a QUAD filling the whole screen.
  9. Render de quad to the default framebuffer.

Is there a way to attach the default GL_BACK buffer to one of the slots of my FBO?

No.

  1. Bind the texture to a QUAD filling the whole screen.
  2. Render de quad to the default framebuffer.

Since you’re using a core context, you could just blit the framebuffer.

See example of using multiple render targets using FBO at Deferred shader. Unless you have a need of complex lightings, etc, you can simply skip the last render stage and do a glBlitFramebuffer() as Alfonse proposed.

[quote]Another way to go is to only use one output of the shader, GL_BACK_LEFT. If you are in picking mode, don’t swap the back buffer with the front buffer. But that would be a problem if you can’t get enough picking information from RGBA.

I thought about using this solution but them I would have to run the vertex shader two times. It is not a problem for my application but I don’t think it is an elegant solution.[/QUOTE]

I don’t think that would be inelegant. You only use the special picking shader when the user clicks on something, otherwise you use the standard shader. This will improve the performance for the usual case (which would be no picking).

I think any professional OpenGL application will eventually end up using multiple shaders for specialized situations.

I am actually using multiple shaders on my application. All my UI elements (buttons, borders, images, etc…) are rendered using different shaders. The problem is that because the way I architectured the UI I can’t do multiple pass renderings so I can’t separate the color picker from the drawing of the elements.

I am almost finishing adding FBO capabilities to my application. If I understand correctly what I need to do to blit is:


Render everything to myFBO
...
glBindFramebuffer(GL_READ_FRAMEBUFFER,myFBO);
glBindFramebuffer(GL_DRAW_FRAMEBUFFER,0);
glReadBuffer(GL_COLOR_ATTACHMENTi); //where i is my desired renderbuffer previsously attached/initialized to myFBO
glDrawBuffer(GL_BACK);
glBlitFramebuffer(0, 0, size.x, size.y,
	   0, 0, size.x, size.y,
	   GL_COLOR_BUFFER_BIT, GL_LINEAR);
...
swapbuffers

Thanks again for the help. I am having really a hard time with the FBOs.

After a lot of debugging I finally got my program running with the FBOs and render buffers. But I am not getting any image on the screen. I checked the FBO for completeness (it is ok) and I am bliting it to the screen with:


	bind(GL_READ_FRAMEBUFFER);
	unbind(GL_DRAW_FRAMEBUFFER);

	glReadBuffer(attachment);
	glDrawBuffer(target);
	 glBlitFramebuffer(0, 0, size.x, size.y,
	   initialPos.x, initialPos.y, initialPos.x+size.x, initialPos.y+size.y,
	   GL_COLOR_BUFFER_BIT, GL_LINEAR);

	glDrawBuffer(GL_BACK);
	unbind(GL_READ_FRAMEBUFFER);

Where bind is a method that binds the FBO to the target and unbind bind the “0” FBO to the target. I am using GL_BACK as target and GL_COLOR_ATACHMENT0 as attachment. Is there something wrong or should I suspect my drawing routines?

Is there an easy way to check if the COLOR_ATACHMENT_i to see if something is written there?

Could someone point me to a minimal code that uses FBOs and render buffers? Idealy doing nothing too fancy like drawing a red triangle to one atachment point and a yellow one to the other, bliting to the default one and being able to switch the bliting at runtime.

Thank you

Finally I got it working!!! I will write a tutorial about it in the morning and post in this thread is case someone in the future have the same problem.

Here is a short tutorial I wrote to help others that could be having the same problem. I would like to thank everyone in the forum that helped me to track this particular problem.

How to get multiple outputs from a shader

This tutorial teaches you how to get multiple data out from a single fragment shader using renderBuffers and a custom FBO. Take into account that this might not be the best way to solve your particular problem, there are other ways of getting the data:

  1. Use multiple shaders each with a single output.
  2. Use the default FBO and the GL_AUXi buffers to get the data.
  3. Use textures attached to the FBO to get the data.

PART0 - glGetError is your friend!

A lot of OpenGL functions can silently throw errors at runtime and you will never know except when you notice that something is not being drawn. I recommend at least putting a call to glGetError in the begging of your render loop in order to get errors from the previous frame.

Since more than one error might have occured you need to call the function on a loop:


GLenum error;
while( (error=glGetError())!= GL_NO_ERROR )
  LOG_ERROR(error);

Check the documentation of the function for more information.

PART1 - The default FBO

Each framebuffer is a collection of images. Each image is attached to an attachement point on the FBO.

The default FBO is the one created togheter with your openGL context. It is used to represent the following images:
1 - The pixels currently on the screen (Attached to GL_FRONT)
2 - The pixels on the back buffer that will be shown when you swap the buffers (Attached to GL_BACK)
3 - A bunch of images related to stereographic rendering (GL_FRONT_LEFT, GL_FRONT_RIGHT, GL_BACK_LEFT, GL_BACK_RIGHT)
4 - Auxiliary buffers (GL_AUXi …)

All those images are created when you create the openGL context and are automatically resized to the current screen widhtxheight.

PART2 - Differences between the default FBO and a custom one

A custom FBO is created by the glGenFramebuffers and destroyed by the glDeleteFramebuffers in the same way as other OpenGL objects.


  GLuint myFBO;
  glGenFramebuffers(1,&myFBO); //Create the FBO
  
  //Code using myFBO goes here
  
  glDeleteFramebuffers(1,myFBO); //Delete the FBO


All the attachement points on a custom FBO have names that are different than the default FBO:
1 - Color attachment points (GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1 …)

Differently than the default FBO, custom FBO’s do not have any of its images created. Those images could be created by a variaty of methods. For this tutorial we will use RenderBufferObjects.

Another important difference is that custom FBO’s are only memory regions and, as so, are not show on the screen. To show it on the screen you need to do one of the following:
1 - Render onde of the FBO attached images to a quad or other geometry (only possible if you attached a texture) and display the quad using the default FBO.
2 - Blit the content of the image to the default FBO using glBlitFramebuffer

Since the custom FBO’s are not managed they do not have their sizes updated with the screen so, if you need to have your FBO matching the screen resolution you need to do this manually by recreating the attached buffers, reseting their storage and finally rebinding them to the FBO.

PART3 - The Render Buffer Object

The render buffer object represent a region on the memory of the video card where you can render pixels. In order to create a render buffer you need to do:


	GLuint myRBO;
	glGenRenderbuffers(1,&myRBO);
	
	//Use the render buffer
	
	glDeleteRenderbuffers(1,myRBO);

The generated render buffers will not have any memory allocated to them. In order to get memory for them you need to set the storage for the RBO with a call like:


glBindRenderbuffer(GL_RENDERBUFFER,myRBO);
glRenderbufferStorage(GL_RENDERBUFFER,internalformat,width,height);
glBindRenderbuffer(GL_RENDERBUFFER,0);

The first argument should always be GL_RENDERBUFFER. Internalformat is the internal image format used to store the data. As an example you could use GL_RGBA32F (32 bits floating point format) or GL_RGBA32UI (32 bits unsigned int format). Look at the wiki for the image formats for more information. Width and Height are the size of the render buffer.

Now you need to attach the render buffer to the FBO.


//First we bind the FBO
glBindFramebuffer(GL_FRAMEBUFFER,myFBO);
//Now we bind the render buffer to COLOR_ATTACHMENT0. 
//You can repeat this to bind other buffers to other color attachments points you need.
glFramebufferRenderbuffer(GL_FRAMEBUFFER,GL_COLOR_ATTACHMENT0,GL_RENDERBUFFER,myRBO);
glBindFramebuffer(GL_FRAMEBUFFER,0);

It is not recomended to reset the buffer storage more than once for each buffer. It is better to delete the buffer, create a new one and reattach it to the FBOs.

PART4 - The shader

To use multiple outputs on your shader you only need to declare them :


/** 
    Sample Frag shader 
    This frag shader receives a color value from the vertex/geom shader and generates two outputs per
	fragment:
    colorOutput
	   - The same color passed by the vtx shader.
	   
    invertedRGOutput
	   - The color passed by the vtx shader with r and g components swapped.
	
	
**/
smooth in vec4 color;
out vec4 colorOutput;
out vec4 invertedRGOutput;

void main() {
	colorOutput = color;
	invertedRGOutput = vec4(color.g,color.r,color.b,color.a);
}

It is important to note that the OpenGL driver is free to allocate the location of colorOutput and invertedRGOutput as it see fit. I recomend to use the “layout” keyword to bind them to especific locations. If you do not want to do it you could:

1 - Use glBindFragDataLocation before linking the program to force a binding.
2 - Use glGetFragDataLocation to figure out where the driver linked your output. (not recommended)

[code:using the layout keyword]
layout(location = 0) out vec4 colorOutput;
layout(location = 1) out vec4 invertedRGOutput;



PART5 - Binding shader outputs to the FBO's attachment points 
After reading this part you will understand why I do not recommend using glGetFragDataLocation.

To bind the shader output to the FBO you will need a set of calls:


glBindFramebuffer(GL_DRAW_FRAMEBUFFER,myFBO); //Binds your FBO for drawing
/**
Shader location 0 will be rendered to GL_COLOR_ATTACHMENT0.
Shader location 1 will be rendered to GL_COLOR_ATTACHMENT1.
**/
GLenum buffers_to_render[] = {GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1}
glDrawBuffers(2,buffers_to_render);

// bind the program and draw here
// it is safe to unbind the program here too (at least for the FBOs :wink: ).

glBindFramebuffer(GL_DRAW_FRAMEBUFFER,0); //Returns the default FBO to the DRAW location



The glBindFramebuffer sets myFBO as the target of every draw operation you want to do. The glDrawBuffers is the call that links the shader locations to the FBO's attachment points. Each position on the "buffers_to_render" vector correspond to a location in the shader. So by calling glDrawBuffers you are saying "shader_location[i] binds to buffers_to_render[i]". So, if your driver decides that a good location for your output is 50 you need to create a 50 elements buffers. If you do not want one of the outputs of the shader simply pass GL_NONE to glDrawBuffers.


/**
Shader location 1 will be rendered to GL_COLOR_ATTACHMENT0. Shader location 0 will be discarted.
**/
GLenum buffers_to_render[] = {GL_NONE, GL_COLOR_ATTACHMENT0}
glDrawBuffers(2,buffers_to_render);



PART6 - Blitting the FBO to the screen / Acessing the data
==========================================================
Ok, so you rendered everything to the memory. Now you need to display it somehow or use your data.

To blit the data to the output you need to set the default FBO as the DRAW_FRAMEBUFFER and your FBO as the READ_FRAMEBUFFER:


glBindFramebuffer(GL_READ_FRAMEBUFFER,myFBO);
glBindFramebuffer(GL_DRAW_FRAMEBUFFER,0);

//Set what color attachment we are reading
glReadBuffer(GL_COLOR_ATTACHMENT0);

//Set to what attachment we will write
glDrawBuffer(GL_BACK);

//This function does the magic of copying your FBO to the default one. 
//You could use it to copy only a region of the FBO to another region of the screen
//The parameter GL_LINEAR is the function to use in case the FBOs have different sizes.
glBlitFramebuffer(0, 0, myFBOwidth, myFBOheight,
   0, 0, ScreenWidth, ScreenHeight,
   GL_COLOR_BUFFER_BIT, GL_LINEAR);

//Resets the Draw Buffer to the default one
glDrawBuffer(GL_BACK);

//Unbinds our framebuffer
glBindFramebuffer(GL_READ_FRAMEBUFFER,0);


To access the data without displaying it on the screen you could use glReadPixels:


glBindFramebuffer(GL_READ_FRAMEBUFFER,myFBO); //binds the fbo
glReadBuffer(GL_COLOR_ATTACHMENT0);
glReadPixels(x,y,1,1,GL_RGBA,GL_UNSIGNED_BYTE,&data); //Call for float data (only use if the render buffer attached to 0 is float)
glReadPixels(x,y,1,1,GL_RGBA_INTEGER,GL_UNSIGNED_INT,&udata); //Call for int data (only use if the render buffer attached to 0 in an integer)
glBindFramebuffer(GL_READ_FRAMEBUFFER,0); //unbinds the fbo


PART7 - A note about image formats
==================================
If you want to use float data types on your shader, your renderbuffers and pixel reading operation need also to operate with float datatypes. The same goes for integer ones. ( I am not sure if this an OpenGL requirement but I can garantee that if you do not do this it will normaly result in something that is not what you want).

I can see you spent a lot of tmie and effort on this. Thanks for sharing, it is what makes the community strong!

One comment: with OpenGL 4, there is a better debugging feature available, as you can get an immediate callback when something goes wrong. This can also include warnings for inefficient data types.

ARB_debug_output is in no way associated with OpenGL 4. It isn’t part of the GL specification; it’s purely an extension and is perfectly capable of being used on non-4.x hardware.

I see, thanks Alfonse!

Thanks. I will take a look at this extension and see if I can get it to work with my logging system.

I have a question/comment about this part of your code,

glReadBuffer(GL_COLOR_ATTACHMENT0);

Does this actually have any effect?

[b]I think when you call glBlitFramebuffer only what is output to location=0 in the shader (layout(location=0 etc)) will be blitted, as far as colour goes anyway.
So if you did the following when rendering


GLenum buffers[] = {GL_COLOR_ATTACHMENT3, GL_NONE, GL_COLOR_ATTACHMENT2, GL_COLOR_ATTACHMENT0};
glDrawBuffers(4, buffers);	

and then blitted to another FBO, only whatever was rendered to GL_COLOR_ATTACHMENT3 would be copied. regardless of what you set glReadBuffer to
[/b]

edit: i spoke to soon, it seems to copy from GL_COLOR_ATTACHMENT0 regardless of the order things are sent to glDrawBuffers and what glReadBuffers is set to.

if you look http://www.opengl.org/wiki/GLAPI/glReadBuffer or here http://www.opengl.org/sdk/docs/man4/xhtml/glReadBuffer.xml it doesn’t mention blitting.