Shader: 2D and 3D arrays Question

[faculaganymede]

Shader Experts.

I am trying to implement a fragment shader that converts the texture RGB to intensity using a look-up table:

RGB -> fragment shader (use look-up table) -> intensity

Note: the intensity value need to be mapped from the look-up table (i.e. it should not be computed in the shader with something like 0.299 * R + 0.587 * G + 0.114 * B).

The look-up table needs to be a 3D array for the 3 color components (i.e. tbl[R][G][B]=intensity). This is what I had in mind of going about it:

1. In application program -> create look-up table -> float tbl[256][256][256]
2. In application program -> call shader programs
3. In fragment shader program -> read look-up table
4. In fragment shader program -> get texture RGB
5. In fragment shader program -> do intensity look-up -> gl_FragColor = intensity

So, I'll need a 3D array variable in the shader program. But, from what I learned, GLSL seems to only support 1D arrays. If I do something like this:
uniform float tbl[256][256][256];

I get a compiler error:
Ready for GLSL
ERROR: 0:4: '[' : syntax error parse error
Link failed. All shader objects have not been successfully compiled."

I came up with another idea, which is to use a sampler2D or sampler3D variable to store the look-up table data, but accessing the data inside the shader seem cumbersome. Does anyone have a better idea?

[sqrt[-1]]

I don't think you could have 256*256*256 number of float uniforms anyway on current hardware.

Your only way to do this is to use a 3D texture (lumance only with nearest filtering) then sample the 3D texture using the RGB vales as texture coordinates.

vec3 rgbvalues = texture2D(srcTexture, texCoord);
float intensity= texture3D(lookup3DTex, rgbvalues).x;

[def]

performance wise, wouldn't a second texture lookup be less efficient than the two instructions (assembly) needed to do the math inside the shader?

[Humus]

Most certainly. This is a typical example of when NOT to use a lookup table. The accesses get extremely random from pixel to pixel ( => very slow) and the math to do the same is extremely simple: dot(rgb, vec3(0.299, 0.587, 0.114)). Plus that it gives more accurate results than the lookup table. To use a lookup table, in particular if you're using wildly varying dependent texture reads, you must replace quite a few ALU operations for it to be a win in performance.

[sqrt[-1]]

While I agree with what you guys are saying, he did actually state that doing the math in the shader was not an option.(I cannot think of a reason why however)


Originally posted by faculaganymede:
Shader Experts.

Note: the intensity value need to be mapped from the look-up table (i.e. it should not be computed in the shader with something like 0.299 * R + 0.587 * G + 0.114 * B).

[funkeejeffou]

Do it oldschool :)

You have an array defined like this : tab[depth][height][width]

tab[i][j][k] = tab[i*width*height + j*width + k]

so define your 3D array as a 1D one and everything will work :
float tab[width*height*depth]

Thus, I recommand you using a 3D texture as the GPU will work in a more optimized way.

Cheers, Jeff.