#include in glsl

[Godlike]

It would be great to have the #include preprocessor keyword in glsl. I know that this requires a few changes in the API but it would be great to have it as an option. I believe that nVidia's cg has it, I dont know about HLSL.

What do you think?

[ruysch]

HLSL has it; For some reason this has never been added to GLSL, I personnally had hoped for at least some sugar in GL3.2 but I dont think you will get #includes in GL any time soon.

I have never really heard a good argument for not providing at least some of the syntetic sugar that both NVIDIA Cg and Microsoft HLSL provides. Perhaps some of the ARB members/contributors could shine some light?

[Dark Photon]

Remote rendering? Too advanced for DX. GL has had it forever. There, the machine the shaders are compiled/used on isn't the machine the shaders are loaded from.

Whether the GLX protocol was ever flushed out to make this happen though, I'm not sure.

You'd think these could be handled on the way in before it hits the server, but consider conditional #includes which depend on state only known in the server's GLSL. Ugly.

That's just my off-the-cuff thought. No doubt there are other reasons for/against.

[yooyo]

Includes is easy job to handle in application. So.. parse your shader code (before you pass string to compiler) and unroll includes as you wish. Driver should not deal with that.

[Brolingstanz]

#pragma escapes macro expansion and the compiler will ignore unrecognized tokens, so it might make for a semi-standard meta-data scaffolding for artist-tool communication, rather than burying and searching for messages in comments or such like.
...
#pragma include("whatsits.h")
uniform sampler2DArray Material;
#pragma attribute(Material, "TMU=0; Filter=LinearMipmapLinear")
...

[ScottManDeath]

Yeah, this is trivial with boost. The added benefit is that you easily can add support for a virtual file system...

std::string Shader::PreprocessIncludes( const std::string& source, const boost::filesystem::path& filename, int level /*= 0 */ )
{
	PrintIndent();
	if(level > 32)
		LogAndThrow(ShaderException,"header inclusion depth limit reached, might be caused by cyclic header inclusion");
	using namespace std;
 
	static const boost::regex re("^[ ]*#[ ]*include[ ]+[\"<](.*)[\">].*");
	stringstream input;
	stringstream output;
	input << source;
 
	size_t line_number = 1;
	boost::smatch matches;
 
	string line;
	while(std::getline(input,line))
	{
		if (boost::regex_search(line, matches, re))
		{
			std::string include_file = matches[1];
			std::string include_string;
 
			try
			{
				include_string = Core::FileIO::LoadTextFile(include_file);
			} 
			catch (Core::FileIO::FileNotFoundException&amp; e)
			{
				stringstream str;
				str <<  filename.file_string() <<"(" << line_number << ") : fatal error: cannot open include file " << e.File();
				LogAndThrow(ShaderException,str.str())
			}
			output << PreprocessIncludes(include_string, include_file, level + 1) << endl;
		}
		else
		{
			output << "#line "<< line_number << " \"" << filename << "\""  << endl;
			output <<  line << endl;
		}
		++line_number;
	}
	PrintUnindent();
	return output.str();
}

[Godlike]

Because of the lack of include in GLSL as well as the capability to have all the shaders code in the same file I've created a parser that supports this features and a few more unimportant at the moment.

The parser works with preprocessor pragmas and the syntax is similar to c++ OpenMP. The useful pragmas are 3:
#pragma anki include "path/filename.glsl"
#pragma anki vert_shader_begins
#pragma anki frag_shader_begins


For example you can write the following (dummy) shader program test.glsl:

// common code
#pragma anki vert_shader_begins
#pragma anki include "test1.glsl"
// vert shader code
#pragma anki frag_shader_begins
// frag shader code

You can feed this to the parser:

shader_parser_t parser;	
parser.ParseFile( "test.glsl" );

...and then extract the 2 shaders code using the vert_shader_source and frag_shader_source:

cout << parser.vert_shader_source << endl;

The above will write:

// common code
#line 2 0 // #pragma anki vert_shader_begins
#line 0 1 // #pragma anki include "test1.glsl"
// Im test1.glsl
// LALAAAAAAA
#line 3 0 // end of #pragma anki include "test1.glsl"
// vert shader code

As you can see it keeps track of the lines so that the driver's compiler can print the correct lines in the error messages and warnings.

Here is the source ancient-ritual.com/programming/shader_parser.tar.gz. If someone is interested I will clean the code, make it completely abstract and properly release it.


PS: I have to mention that the code is not clean because it caries unused code. Code that I use in other libraries of my engine

PS2: It is tested in GCC

[knackered]

I find it difficult to believe that anyone inclined to write OpenGL shader code would be incapable of writing similar functionality in less than 10 minutes. Only use third party code if it's going to save you significant development time - because if it isn't, then writing it yourself will.

[kRogue]

Just a few thoughts on #include in GLSL, the easy part is adding a simple pre-processor to handle #include, then feature creep comes: support #ifdef/#endif guards, then that feature creeps more to having to parse #if/#ifdef/#elif/#ifndef lines... then you start twiggling your head some in that people can make really complicated macro systems too... then you realize to do it the right way, you end up writing a full blown pre-processor.. ick...

In truth it would be a good place for GLU to handle it, along with the idea that you pass to GLU some function pointers for opening files so that you can have your own virtual file system.

Too bad GLU is most likely dead and we will never see it improved (matrix ops, proper GLSL pre-processor, update to GL3 core profiles for GLU tesselators and NURBS). Sighs.

[Alfonse Reinheart]

then feature creep comes

But, since you are implementing this yourself, you can choose not to succumb to feature creep.

Too bad GLU is most likely dead and we will never see it improved

I suppose. But I don't really understand how it could have been updated. It is essentially just a library that makes OpenGL calls. Who would be responsible for distributing it?