(1) If one does not specify a fragment shader, but only a vertex shader, and that vertex shader contains one or more out variables, how does the implementation determine what to do with them?
(2) If one also specifies a fragment shader, and that fragment shader contains one or more out variables, how does the implementation determine what to do with them?
Kip
If one does not specify a fragment shader, but only a vertex shader, and that vertex shader contains one or more out variables, how does the implementation determine what to do with them?
It doesn’t. If you want to use a vertex shader with fixed-function fragment processing, you have to use the fixed-function vertex shader outputs. The colors and texture coordinate outputs.
(2) If one also specifies a fragment shader, and that fragment shader contains one or more out variables, how does the implementation determine what to do with them?
When you set generic inputs for the vertex program, you have to set up a mapping between named inputs in the vertex program and attribute indices. For fragment programs, you do something similar with the outputs. For more details, see this wiki page.
As for (2) if these variables are one of gl_FragColor, gl_FragDepth, or gl_FragData then they are taken by the fixed functionality of openGL to write the final Fragment(that is pixel) values.
For (1) I am not sure. I guess that if they are built-in varying variables like gl_Position then they ought to be picked up by the fixed functionality to perform the per-fragment operations. If they are not then I would assume that the behaviour would be undefined.
I am not 100% certain so you can wait for input from more experienced shader programmers 
Well, the answer depends on what version of OpenGL and what version of GLSL are you using.
The thing is, if the version of opengl that you’re using supports fixed functionality and you’re not overwriting that with either a vertex or fragment shader, Opengl will try to provide the missing shader by the fixed function counterpart. This is very well documented, actually, because it’s … fixed.
So, to give you an example, for GLSL 1.1 (the one that you’re using unless you specify a #version at the beginning of the shader) you can read in this wonderful free spec
http://www.opengl.org/registry/doc/GLSLangSpec.Full.1.10.59.pdf
the entire chapter 7 which specifies all the built in variables that you can acces from either the vertex or fragment shader (yes, that was a friendly RTFM 
). So, if you only write the fragment shader, opengl fixed function pipeline will write most, if not all (depends on what you enable/disable) builtin variables accesible from the fragment shader and you have to output those that they specify there. On the other hand, if you only provide the vertex shader, you will have to output the propper values into the builtin vars necesary for your desired effect to be computed by the fragment fixed pipeline.
On the other hand, for different OpenGL context/GLSL versions, you might be surprised to find out that they either deprecated or, worse, removed those builtin variables and so therefore you won’t be able to use the fixed pipeline in any of the stages. Probably the best way is to just provide both shaders ( you can find implementations of fixed functionality ones, just copy/paste/modify and smile).
Thanks. But what happens if my fragment shader outputs to a variable called “Colour”, not a built-in gl_FragColor. What happens then? Is it assumed that the first variable you output to, regardless of name, as long as it is a vec4 is assumed to be for gl_FragColor?
Kip
Thanks. But what happens if my fragment shader outputs to a variable called “Colour”, not a built-in gl_FragColor. What happens then?
Did you read the wiki page I linked to?
Yes, I did - but I must have missed that part?
Kip
so basically, if you write ONE fragment shader output variable then that is what you get on screen, if you write to more than one, you’ll get one of them, no guarantees.
But this whole discussion is pointless, why fiddle and play around at the outer edge of unspecified behaviour. The quote that Alfonse gave you basically translates to “we, the designers of opengl, encourage the ones who will implement it inside drivers to help you, if you make it clear to them what you want (i.e. if you write one output, that’s probably what you want); but if you play around and are not specific about what you want, the behaviour is implementation specific, meaning it might change from driver to driver”.It’s like me telling someone to cook something and give that person a cook book, even though I expect them to cook something specific and after that being dissapointed in what they choosed. The more cooks, the more different (maybe not what I wanted) dishes…
So just don’t… use on vertex shader, use one fragment shader, specify exactly how inputs and outputs route in and out of your application and probaby the driver will not suprise you…
Alfonse: I saw that, but it still doesn’t answer my question of the default semantics of index 0.
Iulian: Your first paragraph answered one of my questions. Your second paragraph involves a subjective value judgement. The Orange book (3rd Ed) contains examples where only one of the programmable shader interfaces is used and where out variable indices are not explicitly bound.
It’s like me telling someone to cook something and give that person a cook book, even though I expect them to cook something specific and after that being dissapointed in what they choosed. The more cooks, the more different (maybe not what I wanted) dishes…
They’re just numbers. If you don’t choose which fragment output indices map to which fragment output names, the driver simply assigns them. It’s not like there isn’t an API that allows you to query what values were assigned (in case you’re wondering, it’s “glGetFragDataLocation”).
Like I suggested on the Wiki, if you’re using more than one output, then it’s best for you to specify the mapping yourself, since you’re generally going to change programs more often than framebuffers.
I saw that, but it still doesn’t answer my question of the default semantics of index 0.
Zero means whatever color buffer of the currently bound draw framebuffer is connected to the zeroth index of that framebuffer’s draw buffer mapping. This is set with glDrawBuffers, or its more limited earlier version, glDrawBuffer.
Kip, if you can tell me which examples are the ones you’re talking about, I might take a look when I get back home. But you must understand that the examples in that book are made so they show some language features and not always complete implementations/best practices.
Either way, sure, the language allows it, the interaction with the fixed functionality is kinda well documented so in theory you can do it. However, when we translate that into the real world and we factor in the fact that maybe some drivers are less than perfect and maybe they might act up especially in that kind of borderline-behavior, I’d be reluctant to take that path… But if you’re brave enough and find interesting things, please, share your experiences 
Hey Iulian, see pp. 413-415 of the Orange Book, 3rd edition. There may be other examples in there too that I don’t have the time to hunt for. That was the last one I came across.
Hi Kip.
Orange Book, 3rd edition, page 412 :
“We can do this computation just by using a vertex shader. The vertex shader writes to the out variable Color and the special output variable gl_Position, and no programmable fragment processing is necessary. The fragment shader simply passes the in variable Color to the out variable FragColor”
So to me it’s quite simple… they present the full listing of the vertex shader, they tell you that the fragment shader is nothing fancy but still they tell you what it does… so there IS still a fragment shader, but it’s so trivial that if you read until that page, you will know to write it 
I’m preety sure the other examples are like this as well
Hey Iulian, but I don’t think that’s the interpretation they had intended. When they say “no programmable fragment processing is necessary”, they probably mean exactly that - including the trivial case.
C’mon Kip, this is semantics… They tell you what the fragment shader does, it passes the in variable Color ( the IN variable color, but as you can see, in vertex shader it’s the OUT variable color and as you know, basically what goes out of a vertex shader should come in to the fragment shader) to the out variable FragColor. And they tell you that no PROCESSING is necesarry, because in the shader you don’t do anything else other than pass the Color var to the FragColor var (it’s what’s call a pass-through shader).
later edit: in the 2nd edition of the book, indeed, the shader in question was used without an accompanying fragment shader , using the predefined fixed pipeline out variable gl_FrontColor. So, in that case, yes, opengl would take that value and output it, after passing it through the fixed pipeline. (the source code for the 2nd edition is available online at 3dshaders.com).
That is the only shader though that is without a pair, all the others in the pack have a vertex shader and at least a coresponding fragment shader.
Now, I don’t have the 2nd edition book(only the 3rd) to see how it was formulated, but given that it was the only shader without a fragment shader in the book and that he actually explained what the fragment shader should do, I think that it was just a “not so clear” adaptation from the author’s part from the 2nd edition when the predefined vars and fixed pipeline were still not deprecated/removed and the 3rd edition.
Hey Iulian. The last one I found was on p475 (3rd Ed.). Thanks for your help.
It isn’t a particularly well written book, unfortunately. There are typos, incomplete code, and all the omitted parts of the code are the critical stuff necessary for explaining the difference between the traditional fixed function pipeline and the programmable mechanism. To make matters worse, there is no errata available on the website, the website is ancient, and the “source” to the examples is all from the previous edition filled with deprecated calls.
Well, I still believe that the Orange Book is quite good, but it has inerent problems. Also, I strongly believe that if you go for either vertex or fragment shader, go for the whole package. If you don’t, the driver will probably, under the hood, provide the complementary “fixed pipeline” functionality through a full-blown internal shader (that is because noone builds custom fixed function HARDWARE nowdays), and that provided shader will more than likely do some extra work that you don’t want nor need. To keep with the example you gave from the Orange Book (and, for a second, assuming that it would work with fixed fragment pipeline) , I’m preety sure that any shader that the driver will provide won’t be as fast as
#version 150
in vec4 Color;
out vec4 outColor
main()
{
outColor = Color;
}
because it will likely have lots of conditional calculations which you won’t need.
However, if you want to go the “mix and match” way, my earlier recomandation still stands, read chapter 7 from the “choose your GLSL version” specification and intercommunicate with those, hoping that the fixed functionality will do the rest. Just remember that in OpenGL 3.1 “core” profile the fixed functionality is no more and in OGL3.2 the best way to go is to threat it as an extension and prepare for the cases in which it is not supported anymore (highly unlikely yet, but might happen in the future).
keep smiling,
Iulian
Thanks Iulian. =)
Kip