Tangent or object space

I have been trying to get straight what is the point of tangent space and I keep concluding that there is none. The thinking goes like this. Vertex normals sent as attributes to openGL are defined in object or model space; the same coordinate frame as the vertices themselves. They are modified in the vertex shader by multiplication with the gl_NormalMatrix to bend them into eye space. Forgetting the performance issue, it would be equally as fine to multiply the eyePos and lightPos by the inverseModelViewMatrix thereby bending them into object space… the calculations should proceed accurately using the already object space defined normals. If this is the case, then so long as normals stored in a normal map are normals defined in object space, and so long as the normal maps rgba channels simply store this data without referencing it to the width/height or uv coords of the normal map I can see no need for tangent space. If every time the model is moved/rotated the new eye and light positions are bent into model space the normals remain accurate. If every time the model is animated and its texture is warped attached to the vertices at the uv coords as it is, then the pixels within the uv frame are still the same pixels holding the same normals.

What am I missing? I assume that normals remaining in their object space definition are accurately interpolated vertex to vertex accross the fragments and into the fragment shader, and that likewise, lightPos and eyePos if bent into object space would be also via varying vec3’s.

Forgetting the performance issue, it would be equally as fine to multiply the eyePos and lightPos by the inverseModelViewMatrix thereby bending them into object space… the calculations should proceed accurately using the already object space defined normals.

Assuming you don’t have any scale in your matrix, that is correct. If you do have scales, and your lighting computations have a distance factor (attenuation or fog, for example), then you’re scales are going to be wrong. The attenuation distances are going to be off, as will your fog distances.

If they’re uniform scales, you can pass the uniform scale yourself and undo the computations where necessary. If they’re non-uniform scales, you’re in trouble.

If this is the case, then so long as normals stored in a normal map are normals defined in object space, and so long as the normal maps rgba channels simply store this data without referencing it to the width/height or uv coords of the normal map I can see no need for tangent space.

As you point out, that assumes a lot of things.

Let’s take the case of normals in the “normal map” being defined in object space. Doing this means that you need to store 3 components of your normals. If your normals were in tangent space, you would only need two; the third could be computed by a simple rsqrt operation. Tangent space normals can be stored in RG formats (or luminance alpha) to save memory. You can even use RGTC to save even more memory; that’s a 4x memory savings over object-space normals.

And while you might be able to maximize the precision of RGB formats using RGB10A2, I could just as easily use RG16 to get even better precision for the same bitdepth. So either way, tangent-space normals win. They give better precision for the same bitdepth, or they give less memory.

There are other reasons not to use object space normals in normal maps. I’m not sure of the ramifications of doing inverse skinning computations on light positions. I haven’t played with skinning in a long time, but I’m not sure if the transform is fully linear. That is, if none of the matrices have scales, will the transformation through the inverse skinning operations lead to a space with the same scale? If not, then object space lighting isn’t going to work.

I don’t follow your question 100%, but consider this: at some point your model object coordinates must be unwrapped via an atlas of sorts to form uv space. Tangent vectors forward this unwrapping to your shader (e.g. which direction does the green in a normal map apply to in object space). Not sure how you would encode that info. It seems pretty impossible to apply the uv unwrapping frame by frame, that is usually a pretty involved process that is usually done by hand…

Thanks for your replies. Further to Alfonse I would argue that a fully normalized normal in rgb channels may take memory but it relieves computation as the inverseModelViewMatrix is already given. Further, I am developing in Java and every openGL call has to pass the JNI barrier which is a further consideration if vertex attributes are needed to pass in tangent and or bi-tangent data. With skinning i’ll take your word but I am looking at stop frame. Scaling may be a hurdle.

I think nickels is pointing to the root of what I believe I am not understanding. You seem to imply that there is a necessary relationship between the uv mapping and world orientation. But isn’t that just the way tangent space normal maps are encoded? For example, if you generated normals directly from a height map, and you converted the x y slopes offset from z =(0,0,-1) normal to 255/180 (degrees) and stored this information in the rgb channels (the height in the alpha channel for parallax) then texture reads and re-conversion using the standard vertex normals as the poles from which to offset would give you the required modified normals in object space with which to perform the calculations with light and eye also in object space. UV mapping becomes irrelevant.

I don’t suppose anyone knows a good link to creating object space normal maps?

Further, I am developing in Java and every openGL call has to pass the JNI barrier which is a further consideration if vertex attributes are needed to pass in tangent and or bi-tangent data.

Then perhaps you should re-think your API/language choices. And I’m not being flippant. OpenGL itself is defined by drivers with a C interface. If you have to make decisions about your rendering algorithms because your layered API/language imposes an entirely artificial performance bottleneck on you, then you need to rethink whether it is a good idea to use that particularly layered API/language in this instance.

I’m certainly not against Java per se. But I wouldn’t write a rendering engine directly in Java code either, for precisely this reason. It makes more sense to write the rendering engine in C/C++ and then just have Java manipulate that through JNI where appropriate. It makes for a much better abstraction, and you don’t have to base your rendering algorithms on the vagaries of some dubious ideas Sun came up with over a decade ago.

Also, if you’re not using immediate mode, you’re not talking about that many function calls. Just a couple of glVertexAttribPointer calls per object, if that much. And if you are using immediate mode, then you’ve already decided that performance isn’t that important to you, since you’re using the slowest possible method of providing attributes (and across the JNI as well). So I’m not sure how much JNI overhead is really a problem in this instance.

UV mapping becomes irrelevant.

Actually, the UV mapping is far more relevant for object-space normals than for tangent-space ones. The position for each vertex is directly associated with a point on the texture that stores that vertex’s normal. You cannot move this UV coordinate once you have generated the object-space normal map and still expect to get reasonable results.

So texture animation is not possible with object-space normal maps.

Actually, the UV mapping is far more relevant for object-space normals than for tangent-space ones. The position for each vertex is directly associated with a point on the texture that stores that vertex’s normal. You cannot move this UV coordinate once you have generated the object-space normal map and still expect to get reasonable results.

This can’t be the crux of the problem, if you change the vertex position it is still being handed off to the vertex shader with the relevant uv coord attribute, ie, the normal map remains pinned to the correct vertex even though the vertices position has changed, that is, the same normal qua texel read that was at the original vertices position is still at the vertices position after the move. It may be stretched but this is a problem applying to the texture map equally. Stretch aside, if the light vector and eye vector are bent back into object space then the same normal will be at the same vertex and will correctly affect the lighting calculation. That is unless I am wrong which is the question I have been asking from the get go.

I think you’re misunderstanding something.

The foundation of your question is effectively that you want to know why people use tangent-space bump mapping instead of object space. And the basic reason is quite simple: tangent-space bump mapping always works.

We have listed several scenarios under which object-space bump mapping fails. Scales in the model-to-camera transform. It might not work with skinning. There is no ability to animate bump map texture coordinates (potentially quite important for surfaces like water or lava). And that may not be all of them.

Tangent-space bump mapping always works. So long as you provide a proper tangent-space basis matrix, it works. It works with scales. It works with animated texture coordinates. It works with skinning. And anything else that may come along.

They both ultimately do the same job. One can argue texture space efficiency vs. buffer object bandwidth or possible shader complexity or whathaveyou. But in the end, the execution differences between them are going to be rather small.

When you’ve got two options that do the same thing, and one of them doesn’t always work, you’re going to tend to see a lot of recommendations for the one that does. That doesn’t mean that object-space is worthless. But it does mean that you’re going to see more discussion of tangent-space rather than object-space.

In answer to your original question, “what is the point of tangent space?” It works. The scenarios under which object-space fails may not be important to you, and that’s fine for your needs. But those scenarios are likely to be important to others.

That is not the foundation of the question. The foundation of the question is are we getting the right information about object space normal maps or is it like so many other things a piece of misinformation that has circulated for so long and nobody has really stopped to question the assumptions that hold it in place; you only have to look on the web to find x saying rotations are out with object space normals while y is saying they are in. I have stated my assumptions quite clearly, if uv tex coords are tied to specific vertices, and if light and eye positions are converted into object space, then the idea that animations will corrupt object space calculations is a myth… unless there is some factor that I have not seen. I guess there is no factor because none of these assumptions I have repeatedly stated have been questioned, which is what I am really after.

The idea that object space normals cannot be animated is surely false, it would be a matter of applying displacement vectors to the texel fetch over a time function.

Rotation does not change object space, no problem for object space normals.
But as soon as you start to move vertices independently, normals should change, and “applying displacement vectors to the texel fetch over a time function” sound only tractable for procedural animations.
And what are the downsides of tangent space normals anyways ?

If you encode the object space normals as offsets to the standard normals, throw these offsets into the normal map, then decode the offset on top of the standard normal in the fragment shader then light calculations should behave proper. That is, changes in standard normals due to animation would be factored in.

Part of the reason is that I want a development environment for creating assets which enables building normal maps, environment maps, and shader code from basic texture maps and model data, so I can see how the things are looking and am able to tweak them in the same environment. Tangent space mapping is just a whole level of complication that I would rather do without if I do not need it. The more I think about it the more convinced I am it is not needed.

If you encode the object space normals as offsets to the standard normals, throw these offsets into the normal map, then decode the offset on top of the standard normal in the fragment shader then light calculations should behave proper.

I’m pretty sure that doesn’t work. Consider a sphere with a normal map applied to it. Let’s say that the object space normal offset for the left-most point on the sphere is an upwards offset (upwards in object space). If you rotate texture coordinates 180 degrees, the right-most point should have a downward bump. But it doesn’t because the offset at that point is still in the upward direction.

That’s what tangent space bump mapping is all about: being able to properly orient offsets. You need the binormal and tangent vectors to know what direction the offsets go in.

Part of the reason is that I want a development environment for creating assets which enables building normal maps, environment maps, and shader code from basic texture maps and model data, so I can see how the things are looking and am able to tweak them in the same environment. Tangent space mapping is just a whole level of complication that I would rather do without if I do not need it.

I don’t see how tangent space makes this any more or less complicated. The binormal and tangent vectors are taken directly from the texture mapping of the normal map to the surface. They’re easy to compute, as is the tangent-space normal for each point.

Off topic, but maybe you could tell me, what extensions are available to use tangent space with the fixed pipeline?

What I had in mind with offsets was vector addition. Take the example of a floor. Object space normals would all point (0,1,0). If we take a pixel at (2,4) in the object space normal map then it’s normal will be (0,1,0). If we want to perturb this normal 45deg to normalized(1, 1, 0) then we calculate the vector from the top of (0,1,0) to the top of normalized(1,1,0) and store this in the normal map. In the fragment shader we add this offset to the normalized normal interpolated off the model’s vertex normals. Lets say there is a light to the left of the pixel at (2,4). The offset will perturb the normal slope away from the light. Lets move the light to the right of the pixel by spinning the floor about a central pivot. Actually, since the calculations are in object space, the floor hasn’t spun the light has been bought into the fragment shader on the right side of the pixel and the offset now perturbs the normal slope to face the light. So lets bend the mesh and stretch the triangle so pixel (2,4) so it now sits where pixel (10, 5) used in texture space. Let’s say the vertex normals responsible for interpolating the fragment’s normal are also bent so they now face the light, which light is still to the right of the pixel. Assuming the interpolated normal also points toward the light, the offset added to this normal will find the slope pointing 45degrees away from the light as it ought.

Thinking about it, the problem with tangent space from my perspective is that it is a mathematical abstract space. With object space I can envisage where the object is, where the light is, and where I as the viewer are relative to every vertex and fragment and I can imagine how the scene might look. I cannot do this in tangent space either because it is a pure mathematical abstraction like a fifth dimension, or because I have never been able to conceive how I might visualize from within tangent space… I have certainly never read an explanation of how such visualisation is conceivable.

Thinking about it, the problem with tangent space from my perspective is that it is a mathematical abstract space.

Let’s assume that this statement is correct for the moment. That tangent space is abstract and has no concrete visual representation.

Why does this matter? Four-dimensional space time is a “mathematical abstract space”, but that never stopped Albert Einstein from giving us general relativity. Just because something is abstract doesn’t mean it isn’t useful or important. You should not avoid something simply because you can’t picture it in your mind.

As for the accuracy of the statement itself, visualizing tangent space is quite simple.

Take a point on an object. Look straight down on that point, such that the positive axis of the normal is the +Z axis. That, more or less, is tangent space. The tangent and binormal vectors are used to orient the up and left directions of this space, and provide skewing where necessary. And that’s how you visualize tangent space.

Tangent space is nothing more than a convenient way to represent normals without creating an associating between the object that uses it and the normal map itself. Every point of every object has a space tangent to its normal. By storing bump map normals in tangent space, you have the ability to map any tangent space bump map to any surface, without any changes to that object’s vertex data (assuming that they already have an NBT triple) or the normals stored in the texture.

You use probably use tangent space whenever you describe directions on earth like “north” “west” or “up”. These vectors are different on every point on the earth and also change over time as the earth rotates and moves, yet most people find them more convenient than absolute world space vectors.

Excellent and concise explanation mbentrup !

As a philosophical discussion this is getting interesting and I thankyou for your replies… if for no other reasons than I do take on board what is said and am slowly understanding tangent space, but I can only take stuff on board when I can understand it, not when I am simply supposed to accept it.

As an artist I would argue that non-visualisable abstract space is problematic because the visual imagination cannot go there and posit how things might be if the light were a different shade, if it struck the surface at a different angle or with more specularity. Even if tangent space is maplike and therefore visual, it is to the visuals of the object space (probably world space to be honest) the imagination goes when determining these factors… not to the map unless one is following another’s path. From this perspective it makes more sense to accommodate the nature of the mind, especially when providing vector offsets in a normal map will do this. And I can see no reason why such a map should not be applicable to different models because the offsets are always offset from the top of the model’s normals.

As for Einstein, the fourth dimension is time which is why I originally claimed abstraction as being like a fifth dimension. Time, as a fourth dimension is very visualisable… I can visualise how long it will take to get from a to b if I know a and b well, I can visualise how I could hit a tennis ball that came off a server’s racket. Of course Einstein may have mathematised time, but then again, he may have had an intuitive/visual sense of time that he expressed mathematically and a lot of people ran about reading the math never grasping the intuition. I suspect it was the intuition he was trying to express. Indeed, as much respect as I have for Einstein, he was wrong to burden the world with the confidence it has to neglect the visual/feeling state that is our real connection with time (rhythm) in favour of an abstract disenfranchised version owned by only the few.

As an artist

If you’re an artist, why does any of this matter to you at all? Your tool pipeline should be handling all of this for you; you shouldn’t be working with tangent space at all. If you’re being told to deal with tangent space vs. object space, then your tools are terrible and you should get new ones.

I would argue that non-visualisable abstract space is problematic because the visual imagination cannot go there and posit how things might be if the light were a different shade, if it struck the surface at a different angle or with more specularity.

Except that two people have explained that tangent space is not a “non-visualisable abstract space.” Your own unfamiliarity with this space does not make it “non-visualisable.” Or to put it simply, there is nothing abstract about tangent space.

Also, lights don’t have “specularity”. Surfaces determine how much light they reflect, not lights.

And I can see no reason why such a map should not be applicable to different models because the offsets are always offset from the top of the model’s normals.

Which is not surprising, since by your own admission, you don’t understand the mathematics behind tangent space.

“Up” in object-space has a specific direction. And it is always the same direction for each point on the model. If you have an object-space offset that moves a normal “up”, then it is moving the normal upwards in object-space.

Let’s take a sphere with some kind of object-space bump map applied to it. Now, the left-most point has a normal, and the value from the object-space map biases that normal “up”. This up direction is in object-space.

Now, let’s rotate this texture around. Not the sphere itself; the object-space transform doesn’t change. Just the texture coordinates. We will leave the value of the texture coordinate at the point of interest alone, but we’re rotating the normal map by 180 degrees.

Because this point’s texture coordinate didn’t change, the normal it gets will still be pointed “upward”. But we rotated the texture coordinates around.

Remember: a normal map is just a simulation of a bump map (a height field). If we rotate a height-field around 180 degrees, those bumps that were pointed up before are now pointed down.

This didn’t happen with the normals; the normals no longer follow the bumps properly. Object space normals do not take into account the direction of the texture coordinate mapping. Therefore, object space normals are dependent on the texture coordinate mapping; if you change the texture coordinate mapping, you must modify the normals you get based on that.

The technique used to modify the normals based on the texture coordinate orientation relative to object-space is called… tangent space normal mapping. You store the offsets in tangent space. And you provide every point on the surface with a transformation matrix from object space to tangent space.

Time, as a fourth dimension is very visualisable… I can visualise how long it will take to get from a to b if I know a and b well, I can visualise how I could hit a tennis ball that came off a server’s racket.

Fair enough. So how do you visualized curved fourth-dimensional space-time (aka: gravity)? Or time dilation? You can make 2D or 3D approximations, but those are exactly that: approximations.

Please read what I am actually suggesting, for I am not suggesting the use of standard encoded object space normal maps for all the reasons you have outlined Alfonse. The procedure is this:

  1. create the model with standard vertex normals
  2. bake a standard object space normal map from the model
  3. calculate the perturbed normals as theta angles about each pixels referent normal, which referent normal is given by the standard object space normal map; i.e pixel at (4, 6) takes pixel at (4, 6) on the object space normal map as its referent normal. There will be two theta angles one for x displacement and one for y. Also create a height map based upon the height each pixel will be given adjacent pixel’s slope given by the normal’s perturbation
  4. calculate the unit vector of the perturbed normal by averaging the two unit vectors derived from the theta angles
  5. calculate the vector offset from the referent normal to the perturbed normal as a vec3(x, y, z).
  6. store this vec3 in the (r, g, b) of the new normal map. Store the height map in the alpha channel for possible use in determining parallax (this is just an idea at this stage)
  7. in the fragment shader use the standard vertex normals and their interpolations across the fragments as the referents… I assume the object space mapped normals will correspond with the interpolated normals given to the fragment shader, ie, if normal at (4, 6) in the object space normal map is normalized(0.558, 0.008, 0.264), then the fragment shader will give the same normal when interpolating over the relevant vertex normals
  8. use vector addition to calculate the usable normal by adding the offset stored in the new normal map to the referent normal. If the referent normal is unit length then the usable normal will be too

I am a painter artist and I know the value of using the right tools and methodologies to get the results I am after. By ‘right’, I don’t mean standard, I mean those tools and methodologies that give the results I am after are the right ones. I don’t believe tangent space is the right methodology for me. But this somewhat adversarial discussion has been helpful in formulating what it is I am after, so qudos and “not catching butterflys reveals their nature”

The way I visualise curved fourth dimensional space and time dilation is exactly the same way I visualize the time it takes to get from a to b, or the ability to visualise how I could hit a tennis ball that came off a server’s racket. Einstein’s theories are not about some other world I have to go to to see, they are about this world and this world includes these understandings of fourth dimensional space and time dilation.

Tangent-space normalmap: Triangles can share texels. Like tiled detail-textures. Lets you reuse the 128x128 px “brick_norm.png” for all sides of a brick, and will automatically work for all brick-object instances you put to construct a house-object. You can immediately do GL_REPEAT wrapping, to make the pattern repeat 10 times accross 10cm of object surface if you want, the storage requirements remain “128x128 px”

Object-space normalmap: Triangles cannot share texels. Like in lightmaps. Each of the 6 brick faces will need its own texture, sized according to each face’s dimensions. You could cram the 6 textures into one atlas texture. If you were doing the 10 pattern repetitions above, you may need a 1280*6 x 1280 px texture to fit it without loss of IQ.

When you build a house-object of 500 bricks, that do not share orientation, you will need 500 1280*6 x 1280 px textures; or computation in shaders to rotate the texel values. Cool, but what if you want to make a 50m x 5m x5m statue with a similar brick bumpmap? With the 10 pattern repetitions per 10cm. You’ll need a … quite a few gigabytes big texture. And then you will want a column with that pattern…

Normalmaps: just preprocessed bumpmaps. Tilable, compact, absolutely easy to texture with (as an artist). You don’t even need to store a TBN matrix per vertex, only the good ol’ gl_Normal; as you can easily recompute the TBN in either fragment or geom-shader.

Now, if you’re going to bake a normalmap of a low-poly object (30k tri) from its high-poly version (30 mil tri), (every 3D modeling package has this func in either core or as a plugin) then you’ll be baking object-space normals ; though also immediately you’d tick the “bake in tangent-space” option of the plugin. To 1) not need to make an objspace version of the shaders, and more importantly 2) have the resulting texture more easily compressible, with minimum loss in precision. (as the TBN reference frame will contain enough precision as a base, and expected deviations from it are small).
This baked normalmap, even in tangentspace, won’t be applicable to other objects, as it’s not a tilable pattern. If it was meant to be tilable, you’ll know how to process the resulting baked texture to make it tilable. I.e with such a baking you can produce the “brick_norm.png” I mentioned above.

So, basically objectspace normalmaps are completely and utterly useless for artists.