PDA

View Full Version : GLSL Preprocessor



PaladinOfKaos
10-08-2007, 10:06 AM
I'm considering writing a more advanced preprocessor for GLSL (possibly boost::wave if I can re-form it into working with GLSL), adding things like #include, #, ##, #@, and all the other fun preprocessor features that are missing from GLSL. I've got a couple of questions for people, though:
What sort of interest is there for this? If I do write it, what sort of license would people prefer? BSD? GPL? I'm thinking LGPL would probably be best.
There are some implementation issues to work out, but most of them shouldn't be too bad - most of what is #defined by default in GLSL is also available through CPU-side OpenGL calls, so it can be integrated without too much difficulty.

Seth Hoffert
10-08-2007, 11:14 AM
Just as a FYI thing, and not to discourage, OpenGL 3 will add ## and #include.

Korval
10-08-2007, 11:56 AM
OpenGL 3 will add ## and #include.What will #include do? I mean, there's no way that it's going to actually go load a file, so what is its purpose?

oc2k1
10-08-2007, 12:19 PM
You can handle #include with you own preprocessor, because it will be removed.

Currently an important thing for writing a (pre)preprocessor is that the #version line have to be the first line. In that case other stuff like global #defines have to be inserted into the second line or below.

Seth Hoffert
10-08-2007, 12:24 PM
Originally posted by Korval:

OpenGL 3 will add ## and #include.What will #include do? I mean, there's no way that it's going to actually go load a file, so what is its purpose? Good point - I'm not sure how it's going to work, really. :confused:

PaladinOfKaos
10-08-2007, 12:28 PM
the GLSL spec doesn't even mention #include, and states that "There are no number sign based operators (no #, #@, ##, etc.), nor is there a sizeof operator."

Given the huge list of reserved keywords in other parts of that spec, I figured if they were ever going to finish the preprocessor, they would have reserved the directives.

I also think that #include will be much easier to handle client side, unless it's only meant to provide access to standard and vendor-provided libraries.

Seth Hoffert
10-08-2007, 12:32 PM
Originally posted by PaladinOfKaos:
the GLSL spec doesn't even mention #include, and states that "There are no number sign based operators (no #, #@, ##, etc.), nor is there a sizeof operator."
I was referring to the siggraph GLSL pdf - it is mentioned in there.

Seth Hoffert
10-08-2007, 12:35 PM
My bad. I see what you were getting at now. :D

skynet
10-08-2007, 03:12 PM
It might not be a bad idea to actually add #include and implement it via a callback mechanism. What do you think?

PaladinOfKaos
10-08-2007, 03:54 PM
reading through some of nvidia's docs, #include in CG uses the application's current working directory as the search path. That might be what GLSL will do.

Korval
10-08-2007, 06:51 PM
That might be what GLSL will do.I sincerely hope not.

The last thing I want OpenGL to start requiring of implementations is file access. Make it a callback or something, but absolutely do not let the implementation think that it is allowed to start rummaging around my harddrive.

Rob Barris
10-08-2007, 10:35 PM
There won't be any file access implied in the GL3 mechanism.

What you will be able to do is submit buffers with text in them, and attach names to those buffers. Then you will be able to compile some text that has #includes in it, and they will get expanded as needed.

This is the way you have to do it, to preserve client/server semantics. You can't really do callbacks in a client/server model.

Jan
10-09-2007, 02:45 AM
Nice idea.

skynet
10-09-2007, 04:07 AM
What you will be able to do is submit buffers with text in them, and attach names to those buffers. Then you will be able to compile some text that has #includes in it, and they will get expanded as needed. But this would require the application to upload all and every source code that _might_ be required. In turn, this requires the app to know beforehand what #include's are used. This would require us to parse the files ourselves... which eventually leads to a complete preprocessor on our side?!

As a sidenote, it would be nice to be able to feed the GLSL preprocessor with #define's as well (programatically). This would be nice, since we could enable/disable codepaths this way, depending on application-provided symbols.

knackered
10-09-2007, 07:22 AM
yes i agree with that last request. I can define pre-processor definitions in my CPU compiler, so why not my GPU compiler?

Korval
10-09-2007, 12:02 PM
But this would require the application to upload all and every source code that _might_ be required. In turn, this requires the app to know beforehand what #include's are used.... and? I mean, don't you know what and where your shader "headers" are? Can't you load them all beforehand and give them all to GL?

The point of the include mechanism is to allow you to be able to connect with code from shader source files without bothering the client about it directly. That is, without having to add an extra string from the client code, and thus requiring the client code to magically know that this shader file needs the contents of some other shader files.

The point is not for the client code to be totally ignorant of what the shader source might ask for.

Think of it as providing to GL3.0 the filesystem that it searches over. Just like you have to tell your compiler where to search for headers. You just have to tell it in a different way.

PaladinOfKaos
10-09-2007, 01:57 PM
That's not a bad design, actually.

And really, how many "headers" will most projects actually have? 10? 20? Having to load the files beforehand isn't really a big deal. And once they're handed off to the driver, you don't have to worry about them again. It's not really a huge deal.

PaladinOfKaos
10-11-2007, 07:57 AM
Alright, I'll hold off on implementing anything until GL3 comes out, and I can see how it works. I may simply write a scanner that searches for #includes, and loads the proper files.

One other thing that I might try to add support for, if it isn't in GL3, is an equivalent to the "-D" switch of compilers - that is, a compile-time constant definition. With GL3s new uniform handling, that would give us the ability to either pass in data via uniform buffers, or define it at compile time.