View Full Version : Computer architecture perspective of GLSL run time compilation of shader sources

06-03-2013, 03:33 AM
I am trying to understand (I am a math major so forgive me) from a computer architecture perspective why it is that the newer OpenGL movements compile shader sources at run time. What is the motivation behind this? Using basic knowledge from a computer architecture course I took, this is my understanding:
Every GPU that has an OpenGL implementation could potentially have a different instruction set architecture for each of its shader cores, so it does not make sense for a portable graphics library to rely on prebuilt binaries. Rather, compile the sources at run time, this way the driver for each GPU will take care of the compiling and thus shader code becomes portable.

Does that about sum it up or are there other reasons on top of this?

06-03-2013, 05:22 AM
D3D has supported binary shaders since pretty much forever by using a two pass compilation model.

The first pass takes the high level shader source code and converts it to hardware-independent byte code. This is purely a software-only compilation/conversion pass, does not interact with the GPU or it's driver, and can be done off-line. The byte code can be saved out to a binary file and reloaded from that file for subsequent use. The compiler used for this pass is provided by Microsoft as part of the D3D runtime, although - in theory - and if you had the byte code specs - you could write your own (or even compile any other language to the same byte code format).

The second pass takes that hardware-independent byte code and translates it to a hardware-specific representation. This is the actual shader object used for drawing with, and this is the part where the shader gets validated against device capabilities (although there is some pre-validation in the first pass too) and where actual portability between different vendor's hardware is a concern. The compiler (or converter) for this pass is provided by the hardware vendor as part of their driver.

None of this is theoretical - this is a shader compilation model that is out there, right now, being used in real programs that are being used by real people. It's important to keep that in mind when considering any objections - it's not a matter of "will it work?" because it does work.

There's no purely technical reason why GLSL couldn't have (or have had) a similar model, but for whatever reason, it doesn't.

There are some advantages to the GLSL model that the D3D model doesn't have, but whether or not those advantages mean enough to outweigh the disadvantages is a matter I won't comment on. One obvious one is that the D3D model cannot perform any hardware-specific optimizations during it's first pass; an example might be "don't branch here, do branch there, because I know that's the faster way on this hardware". Another is that you must specify a target shader profile up-front and during the first pass with the D3D model (the second pass either succeeds or fails depending on whether the shader will actually run on the hardware), whereas with GLSL you can select to gracefully degrade to meet actual hardware capabilities (by e.g. supplying different fragments of source code to your glShaderSource call as appropriate).

Finally, it's important to discount the whole "protecting shader source code" versus "making shader source code available" argument. Even with the D3D model, if you have the byte code specs you can write a proxy DLL and reverse-engineer. Hell, you could even just run a program in PIX and view an assembly translation of the byte code. It's not about that at all - it's about trading off the extra flexibility that the GLSL model offers versus the ability to move the (in theory) much slower part of the compilation process offline that the D3D model offers.