id's "Rage" engine, Tech5 is OpenGL based

[Korval]

what is the advantage (in OGL 3.0) of having the possibility to combine/attach several (compiled) shaders into a single program object?

Each shader doesn't have to be a complete piece of a programmable component. It's the same reason you don't necessarily put all of your C/C++ code in one file.

A shader can be a set of utility functions that are included in every program you link. You then let the linker's dead-code elimination remove the stuff it doesn't need. And, in a reasonable implementation of this construct, you won't need to do the compilation work multiple times.

But that being said, the translation from an imperative bytecode to SSA form takes only a small fraction of the compile process, so it really isn't worth it.

That kinda begs a question.

If we have the following compiler stages:

• Compile source into SSA
• Perform dead-code removal/inlining
• Convert SSA to machine code

Where is the glslang compiler performance going? I mean, is compiling a C-like language into SSA-form really that time consuming? Even my fairly old computer can compile a several-thousand line .cpp file in less than a second, and it's doing optimizations, inlining, and all sorts of other stuff. Obviously if I start instantiating a bunch of templates, it takes longer, but straight C++ is pretty fast in terms of compilation. And glslang is much simpler.

Is it the dead-code removal?

Or is it simply that IHVs haven't prioritized the performance of their glslang compilers? I mean, we all know about nVidia's silly "dobule-compile" in their glslang. But is it simply that all IHVs have one part-timer working on their glslang compiler, such that after 2-3 years they still don't have decent implementations or compiler performance?

Maybe we should find some way of putting pressure on the IHVs. I mean, GL 3.0 has to do that, since glslang is required to do anything. But GL 3.0 adoption will be impacted by the quality of 3.0 implementations. And the quality will be impacted by the adoption. Etc.

Just for the sake of correctness, none of the above is true.

Such a fine argument you have made

If what you say is true, then GPU's are in fact functional programming devices. Also, doing register allocation in C-style parsing is not a solved problem. And what Zengar was talking about is not functional programming.

[Zengar]

Actually I was not going to comment on it, but...

Originally posted by Korval:

If what you say is true, then GPU's are in fact functional programming devices.

GPU shaders are more "functional" then imperative. A shader is a pure function with no side effects at all. Modern GPUs are of course imperative but we were not talking about GPUs, we were talking about graphics (and shading languages).

Originally posted by Korval:

Also, doing register allocation in C-style parsing is not a solved problem.

No idea was parsing of C-style code has to do with register allocation. But yes, register allocation is a much-researched topic and is, as the matter of fact, very effective with SSA-like form I was talking about.

Originally posted by Korval:

And what Zengar was talking about is not functional programming.

No it was not. I was talking about declarative (at least something like it) programing. A feature that actually marks functional programming is the presence of higher-order functions.

[Overmind]

Where is the glslang compiler performance going?

I can't say for sure for the GLSL compiler, but generally most of the compile time is spent on various optimizations.

For example register allocation can be a very expensive operation, depending on how many registers you have and how "optimal" the solution should be. Also, there may be other optimizations going on, like arithmetic transformations, various forms of code motion, or transformation of a sequence of simple operations into more complex operations (vectorization, ...).

I don't know if current GLSL compilers are doing any of these, but all of this can be very costly.

Even my fairly old computer can compile a several-thousand line .cpp file in less than a second, and it's doing optimizations, inlining, and all sorts of other stuff.

Personally I never experienced any problem with shader compilation time, but I can see that when compiling some thousand shaders every millisecond becomes important. Especially when you can't load all shaders at application startup, and can't accept any variation in framerate (e.g. VR applications, where users get sick when you have too much lag ).

[knackered]

I was just wondering, and seeing as though this topic has morphed into a discussion of GLSL compilers, could I get a general consensus on something?
Can we completely rely on the 3 main GLSL compilers (nv,ati,intel) removing functions/variables/uniforms/varyings that don't contribute to the final output?
Is there something in the spec, or is it something anyone's observed to always be true in practice?

[Jan]

If i learned anything important about OpenGL in the past years, it is not to rely on ANYTHING and not to be surprised about the weirdest behavior. I fear that with such a complex part of OpenGL (GLSL) all your hopes are in vain.

Jan.

[knackered]

thanks for your reply, but no - I know the nvidia driver definitely removes uniforms that don't contribute to the final output (because you can't retrieve their location in your app), but I was wondering if anyone had maybe benchmarked a shader that calls a hugely complex function that doesn't contribute to the output, and noticed no difference between a build with or without it. I'd like some feedback for hardware other than nvidia, to be honest. If needs be, I'll have to run another pre-processing pass over my shader code to remove such things (variables and functions that either aren't used or don't contribute to the output in some other way). I don't want to write that pre-processing step, but if I have to I will.
P.S. I know you can't rely on anything, which is why you should go off the mean average approach adopted by the 3 main IHV's. It's all you can do. I think it would be madness for a driver not to do this optimisation - if you've gone to the trouble of writing a compiler, this would be a trivial thing to add.
Having said all that, it's occurred to me that they must work backwards from the outputs as they compile to assembly, so it would be a natural side-effect of this process to eliminate dead code and variables.

[Lindley]

I assume you're using generic shader code which will be compiled to a number of more specific shaders, where various portions may or may not be used?

I've heard that shaders can understand #ifdefs, although I haven't tested that myself. And it might only be Cg. You'd have to experiment. If so, that would be the easiest way to get guarantees.

[knackered]

Yes I know about #ifdef's, and yes they can be used in GLSL along with #if's, #elseif's, and #define's.
There are various reasons why #ifdef's will be a bit impractical for what I have planned - and what I have planned will be very, very cool.

[Korval]

Can we completely rely on the 3 main GLSL compilers (nv,ati,intel) removing functions/variables/uniforms/varyings that don't contribute to the final output?

Well, we can probably assume it will. With all the inlining and so forth that compilers have to do, it'd be pretty hard for them to include functions that aren't included.

Now, what constitutes "contribute to the final output" is probably going to be something of a problem. If you include a function you don't call, I'd be willing to assume that it will be culled. But if you include a function that you call, but through whatever logic you use doesn't actually contribute to the output, I imagine it will not be culled.

[knackered]

Good point - everything's inline'd on all hardware up till now, isn't it? Even if they aren't, if i don't call them all they're costing is memory and enable speed I suppose....what am I worried about?

There's not going to be any attrib/variable/uniform driven conditionals, I'm pretty sure I'll be avoiding them for a good while yet.
But maybe yes it would be a problem if some getScale() function always returns 1.0 because of a #define - I can't realistically expect the compiler to optimise out a statement that multiplies by the result of a function that always returns 1.0. Or could I?