ARB Fragment Program

[Ysaneya]

There is definately more than 8 bits of precision for tex coords, and they can be signed.

Are you 100% sure your lack of precision is coming from passing the normals onto the pixel shader ? Ie. you do not suffer from a lack of precision in your "norm" array before ?

Are you running your color buffer in 32 bits ?

Y.

[maximian]

I think the texture coordinates, like color are clamped to [0.0-1.0] are they not? If this is not the case, I can change. However, that would make little difference.

As to the normal precision, I used 3 float(32 bit) to store the normals. If I pass using the regular glNormal3fv, then I can "see" that the normals are rendered correctly and I am not losing any precision. However, as I said, If I pass the normals via glColor3f, or glTexCoord3f, I seem to lose precision.

Here is the cg program

struct IN
{
float3 normal : TEXCOORD0;
};
 
struct OUT
{
float4 color : COLOR;
}
 
OUT main(IN i)
{
   OUT o;
   float4 viewer = float4(0,0,1,0);
  float4 red = float4(1,0,0,1);
  float4 blue = float4(0,0,1,1);
  float def = 1.0;
  float epsilon = 0.0005;//can be zero for 
  //exact  match
  float4 v1,v4;
  float dotP;
 
  v1.xyz = i.normal*2;
  v1 = v1-1.0;
  v4 = v1*viewer;
  dotP = v4.x+v4.y+v4.z;
 
  o.color = ( abs(def-dotP) < epsilon ) ? red : blue;
 
  return 0;
}
 
I also tested by passing just the dotP, and it seems like there is a lack of precision. Ie areas with different dotPs are shaded exactly the same.
 
I did that by doing 
o.color = float4( dotP, dotP, dotP, 1);

Thanks for the help

[idr]

The interpolated values of texture coordinates are not clamped. Clamping may happen when you use the textures coordinates to sample a texture (depending on the texture wrap setting). It works more like positions than colors. The value can be anything, but it may be "clipped" when it's used.