Normal Mapping with TBN Matrix does not work properly (yet)

Hi all,

I work on a private OpenTK (C# wrapper for OpenGL) project. I set up a cube that has some texture on it. I also load a second texture that acts as the normal map. I generated the tangents and bitangents for each of the cube’s faces (I checked if they are correct, and they seem to be).

I send vertices, texture coordinates, normals, tangents and bitangents to my vertex shader:

#version 330
 
in		vec3 aPosition;
in		vec3 aColor;
in		vec2 aTexture;
in		vec3 aNormal;
in		vec3 aNormalTangent;
in		vec3 aNormalBiTangent;

out		vec4 vPosition;
out		vec4 vColor;
out		vec2 vTexture;
out		vec3 vNormal;

// for shadows:
out		vec2 vTexCoordinate;
out		vec4 vShadowCoord;

// for normal mapping:
out		mat3 TBN;

uniform mat4 uMVP; // model view projection matrix
uniform mat4 uM; // model matrix
uniform mat4 uMV; // model view matrix
uniform mat4 uNormalMatrix; // inverse transpose of model matrix uM
uniform mat4 uShadowMVP;
uniform int uUseNormalMap;

 
void main()
{
    vShadowCoord = vec4(uShadowMVP * vec4(aPosition, 1.0));
	vPosition = uM * vec4(aPosition, 1.0); 
	vColor = vec4(aColor, 1.0); 
	vTexture = aTexture; 
	vNormal = normalize(vec3(uNormalMatrix * vec4(aNormal, 0.0)));
	
	vec3 tangent = normalize(vec3(uM * vec4(aNormalTangent, 0.0)));
        vec3 biTangent = normalize(vec3(uM * vec4(aNormalBiTangent, 0.0)));
        vec3 normal = normalize(vec3(uM * vec4(aNormal, 0.0)));
	TBN = transpose(mat3(tangent.xyz, biTangent.xyz, normal.xyz));
	
	gl_Position = uMVP * vec4(aPosition, 1.0); 
}

I will not paste all of the fragment shader because it is quite large. The important part consists of a loop that iterates over 10 lights (maximum) and calculates the diffuse light component for each. The following snippet is from / around that loop:


        // ...
        vec3 normal = vec3(0, 0, 0); 
	if(uUseNormalMap > 0)
	{
                // if the texture has a normal map, use the normal from the normal map instead:
		normal = texture(uTextureNormalMap, vTexture).xyz * 2.0 - 1.0;
	}
	else
	{
                // else use the normal that I got from multiplying the object normal with the inverse transpose of the model matrix
		normal = vNormal;
	}

	vec4 colorComponentTotal = vec4(0,0,0,1);
	for(int i = 0; i < 10; i++)
	{
                // if uLightPos[i].w is -1, the light does not exist, so we can end the loop with a break;
		if(uLightPos[i].w > -1){
                        // lightPos is in world space:	
			vec3 lightPos = vec3(uLightPos[i]);

			vec4 lightColor = uLightColor[i];
			vec3 lightTargetPos = vec3(uLightTargetPos[i]); // if it is a directional light, it has a target position
			vec3 lightVector = lightPos - vec3(vPosition);
			float distance = length(lightVector);
			
			lightVector = normalize(lightVector);
			float dotProductNormalLight = max(dot(normal, lightVector), 0.0);

			if(uLightPos[i].w > 0) // is it a directional light?
			{	
                                // calculateDiffuseComponent calculates the amount of diffuse light depending on the distance between light and fragment
				diffuseComponent = calculateDiffuseComponent(dotProductNormalLight, distance, 8.0);
                                // calculateFallOff calculates the light cone for directional light. 
				diffuseComponent = calculateFallOff(diffuseComponent, lightVector, normalize(lightTargetPos));
			}
			else
			{
                                // calculate the diffuse component for a point light depending on its distance to the fragment:
				diffuseComponent = calculateDiffuseComponentPoint(dotProductNormalLight, distance, 2.0);
			}
			colorComponentTotal = mix(colorComponentTotal, lightColor, min(lightColor.w, diffuseComponent));
			diffuseComponentTotal += diffuseComponent;
		}
		else
		{
			break;
		}
	}

	outputColor = (diffuseComponentTotal * mix(colorComponentTotal, vColor, 0.1) + ambient * vec4(1,1,1,1)) * texture(uTexture, vTexture);

The problem is that I still seem to confuse the different spaces. The TBN was built by multiplying the model matrix with the object normal, tangent and bitangent. Then it got transposed (see vertex shader code).
My understanding now is, that this TBN can convert any object from model/world space to tangent space. Or is it the other way round? What do I have to do with the normal that I get from the normal map? Do I have to multiply it with the TBN matrix?

Thanks in advance!

I think I may have fixed it myself. My solution to this post is also on:

Hope this might help others. I think the hardest part is either converting everything to tangent space or just converting the normal from the normal map texture to world space (which I did).

Cheers!