Dumping ObjectCode from Shaders??

[evanGLizr]

Originally posted by Jan:
No, you wouldn't need to detect the hardware everytime.

I would do it this way: Ship the app without precompiled shaders. When the app runs, send the shader to the hw, get the compiled binary, store it on disk. The next time you run the app, check, wether the binary is available, already, and send that instead. If the hw complains, just send the original shader.
[...]
Jan.

The driver can already do that without the need of new extensions, there's no reason why it cannot hash the shaders and persist them on disk.

[jide]

The idea seems interresting. However, I wonder how can this be achieved. Let me try to explain more why I questionate myself.

Who can garantee that a 'compiled binary' shader has exactly the same bits from one card to another one ? Maybe all geforces can have the same binary for a shader, but I think the main problem is to reuse a code compiled by (or for) an Nvidia card under let's say an ATI one. Virtual machines might not behave the same, more optimizations might be really different, as they strongly rely on hardware.

I might be wrong, but at least that would help me understand why.

[ZbuffeR]

To Jide, V-man, and others :
It is NOT about shipping compiled GLSL code, nor dealing with a common binary code across multiple GPUs.

evanGLizr, you have a good point. But it means more hassle on the driver side.

[Jan]

jide: Read my and ZbuffeRs posts again. We do agree with you, that it wouldn't work that way, but our idea is different.

[Korval]

The problem with your idea comes from when the user upgrades his machine. Obviously upgrading from an ATi card to an nVidia card is going to cause all kinds of problems, but even upgrading in the same family tree will create issues.

Another problem is not getting driver optimizations. If you have cached a copy of a relatively unoptimized binary, and the user installs a new driver, you won't know to recompile the original shader.

[Jan]

I don't see, where there are any problems:

If you install another card, changing from ATI to nVidia, or vice versa, the driver will dismiss the precompiled shader as invalid. Of course, each vendor needs to put a small header into the shader, to be able to validate the shader.

Additionally, the driver can put in a version-number, the shader has been compiled with. So, if i install a new driver, the next time i run my app all shaders will be "invalid", because the driver knows, that the shaders have been compiled with an older driver and therefore need to be recompiled.

So, WHERE is the problem??? If you change your hardware, your shaders get recompiled. If you install a newer (or older) driver, that has an improved shader-compiler, your shaders get recompiled. EVERYTIME something important happens, your shaders get recompiled.

BUT, how often do you change hw or your driver? Maybe once a month, you upgrade your driver. So, ONCE A MONTH you pay the additional cost to compile all the shaders your apps use. In contrast to EVERYTIME YOUR APP STARTS, this might be a nice improvement and it is easy to implement, on both sides, the application AND the driver.

Jan.

[jide]

First, sorry Zbuffer, I knew this wasn't the question, but I think my post is, anyway, relevant. If you think not, please do not care or tell me if you think a new thread should be best.

Then, for the last post of Jan. This is certainly right (thought I'm not a 'hardware-man'). However, for people that wants to hide their code, this is still a problem: shaders sources need to be present each time the program has to recompile, this turns out. However, as some would still appreciate it, I agree too: most of the times, when you start a program using shaders, it wouldn't spend much time during loading process... this will indoubtly be appreciated by the end-user.

Now, to try to develop more about that. As everyone knows here, any program (like binary) could be looked up 'cowardly'. I mean one can do reverse-engineering on a binary to study it: this is the main way crackers crack applications.
So, having the shader beeing compiled each time a program first need it, ensures that (at least enforces that) the compiled code wouldn't be seenable by people. Of course, we could try to read directly the graphic memory, but I think it's very more dangerous than if the code were directly readable from a file stored on an hard-disk.

So, I finally wonder how far this would be a so great capability: compiling shaders is not like compiling a full linux kernel

Hope that's not too bad

[Jan]

Well, there are always ways to crack an application. You can't do anything about that. I mean, i can encrypt my files, but if you want the shaders, glIntercept can give you the code. So, if you want to be safe, you would need to send encrypted shaders to the gfx-card. Urgh, bad thing!

And with precompiled shaders it would be the same thing.

And, well shaders are not so long or complex, usually you know, what a shader does, anyway, so you won't be hiding real secrets there.

If you are talking about CRACKING, well then you would need to make sure that a program doesn't REPLACE a shader. That means, you want to bind a lighting-shader, and the crack simply binds something different.

So, if you are concerned about security, forget it. There is no way, you can make this safe and any approach to implement anything security-related into OpenGL is a waste of time.

Therefore this whole discussion has quickly been reduced to the problem of speed.

And yes, compiling shaders does not take that long. But in coming titles more and more shaders will be used, so the impact of shader-compiling will get stronger. I use 3 different shaders in my app, but my code selects only relevant parts of the shader and therefore disables/removes unnecessary code. This way i already get around 20 different shaders, at the moment, that are actually in use. Although they all are built from only 3 different base-shaders. I don't want to know, how many shaders the Unreal 3 Engine uses.

This is one (IMHO promising) idea how to solve this problem. It is all about improving speed at startup and therefore it is all about satisfying the end-user. Nothing more. It's not a necessarity, it would be a nice feature.

Jan.

[DieselGL]

Hi People!!

Sorry for my late answer, I was kinda busy the last week.

Thanks for your very interesting posts and the discussion.

I agree that there should be a way to save the shader once it has been compiled on the users computer. This shouldn't be to difficult for ATI or NVIDIA to realize that, I think. When the user changes the GraphicsCard - just recompile and everything is ok. Perhaps someone is picking up this idea...

@obfuscating

I save m Shaders in an xml File. Well, I could put it in a encrypted zip, but I think I will just use a simple encryption like Cesar's method. Or is there already a Tool ( perhaps a powerful one, means opensource ? ) for that???

rya.
DieselGL.

[yooyo]

Someone can put opengl32.dll into app dir and intercept all OpenGL calls. In one of this call's will be your shader soruce.

Accidently I found that using CG shaders in separate thread and context cannot be intercepted using glIntercept. I didn't try glDEBuger.

yooyo