PDA

View Full Version : Translation from HLSL to GLSL - weird light calculation



Vikus van de Merwe
03-30-2017, 10:16 AM
Hello,

I'm currently working on a translation of a HLSL shader to a GLSL shader. The HLSL shader is a shader for Total War: Attila, provided in the official modding tools and useable in the 3ds Max viewport. I have started working on a translation from the HLSL file to a GLSL file which can be loaded as a custom shader in Substance Designer 5.6.2 to create a better synchronized workflow during the texture creation process.

In the beginning I had some application-specific problems which I managed to solve with the help of the Allegorithmic product support: https://forum.allegorithmic.com/index.php/topic,16074.msg67636.html

Now I'm at a point where all base functions are implemented and useable in the GLSL shader but I'm really new to shader programming (however, I think I learn fast because of my advanced knowledge in programming itself). The initial HLSL file contains about 2500 lines of code. So that doesn't make things easier unfortunately. All in all, I went through the whole Direct3D code and translated it to the equivalent OpenGL statements. I had to fix some matrix calculations and some minor other things but the the GLSL shader is already capable to read all possible kinds of game textures. In addition the TBN matrix (which actually is a mat3(tangent, normal, bitangent) matrix) should work fine as well as the normal map is represented correctly in Substance Designer's 3D view.

However, the core problem now is that I get completely different results with the original HLSL shader in the 3ds Max viewport and the GLSL shader in the Substance Designer 3D view. The light calculations are exactly the same in both shaders (at least as far as I can tell, it might be possible that I messed something up during the translation).

Here're some pictures to visualize my problem (please have a look into my following posts, I never posted in this forum before so it prevents me from linking multiple pictures).


It looks like my GLSL shader doesn't really calculate any specular color (Total War: Attila uses RGB specular maps) and for some reasons the light calculation looks broken.

Maybe someone can give me more specific hints what the problem might be? I can attach parts of the source code or the whole one here as well but I don't know if it's a good idea since it has more than 800 lines of code right now. Anyway, any help is appreciated and if you need more informatin I will provide it as soon as possible!

Dark Photon
03-31-2017, 06:40 AM
Feel free to post short code snippets and ask specific OpenGL questions here. Just make sure you 1) have the rights to post the source code you're posting, and 2) you post the source code inside ... or
... blocks so that it's easier for folks to read.

Generally avoid posting large amounts of course code or asking broad "please fix my program for me" questions. See the Forum Posting Guidelines (https://www.opengl.org/discussion_boards/showthread.php/176139-Forum-Posting-Guidelines?p=1230021&viewfull=1#post1230021) for more details.

Vikus van de Merwe
03-31-2017, 11:15 AM
Thanks for your reply. For some reasons I can't link any picture in my posts because the forum doesn't accept them.

Anyway, I was able to fix the problem myself. The problem was that I didn't convert the textures into linear RGB space before the calculations. 3ds Max does this automatically when loading textures. However, in Substance Designer I have to convert the textures from sRGB space to linear RGB space manually in the fragment shader.
For the sake of completeness I would have linked some pictures to visualize the problem I had but it looks like I don't have the rights to do so at the time.

Dark Photon
03-31-2017, 07:23 PM
However, in Substance Designer I have to convert the textures from sRGB space to linear RGB space manually in the fragment shader.

Can't you just use sRGB GL textures? Then there's no need for manual conversion.


For the sake of completeness I would have linked some pictures to visualize the problem I had but it looks like I don't have the rights to do so at the time.

Just post a few URLs without the leading http:// (possible adding a space or two) and I'll come along and fix them up for you.

Sorry about the inconvenience of not being able to post images yet. After posting a few more messages to the forum, you'll be able to without any help from a moderator. This is a spam prevention feature in the forum software to stop very new users from posting ad links to their products or web sites.