Alising artifact on cutoff image

[clementchu]

Hi,

My render object is a volume cube (350 x 350x 60). It is composed by 60 slices of area 350 x 350. Each slice can be visible or invisible. If all slices are visible, there is no aliasing artifact. If the cube is 20-30 slices visible, there is aliasing artifact on both surface of 20th and 30th slices. My shading code is using ray cast method. I think the solution is to find out the texel center and start ray cast on that texel, but I have no idea how to find the texel center. It is appreciated if anyone can suggest how to do or suggest other method to avoid aliasing artifact. Thanks.


Regards,
Clement

[Dark Photon]

Lets see your shader code for computing the texcoord and doing the lookup, as well as a pic of the artifact. Also, do you have MIPmaps on your texture, and what MIN and MAG filter are you using?

Curious how you are making the slices invisible. If with shader branches, suspect that.

[clementchu]

Hi Dark,

There are vertex, fragment codes and screen shots of aliasing artifacts as below. I haven't use mipmap on texture and I set MIN and MAG with GL_NEAREST.

// vertex
varying vec4 cameraPos;
varying vec4 vertexPos;
varying mat4 texgen;
void main(void) {
    vertexPos = gl_Vertex;
    gl_Position = ftransform();
    cameraPos = gl_ModelViewMatrixInverse*vec4(0,0,0,1);
    texgen = mat4(gl_ObjectPlaneS[0],
                    gl_ObjectPlaneT[0],
                    gl_ObjectPlaneR[0],
                    gl_ObjectPlaneQ[0]);
}
 
 
// fragment
uniform sampler3D baseTexture;
 
varying vec4 cameraPos;
varying vec4 vertexPos;
varying mat4 texgen;
void main(void)
{ 
    vec4 t0 = vertexPos;
    vec4 te = cameraPos;
 
    // after normalize
    t0 = t0 * texgen;
    te = te * texgen;
 
    const float num_iterations = 512.0;
    vec3 deltaTexCoord=(te-t0).xyz/float(num_iterations-1.0);
    vec3 texcoord = t0.xyz;
    vec4 fragColor = vec4(0.0, 0.0, 0.0, 0.0);
 
    while(num_iterations>0.0)
    {
        vec4 color = texture3D(baseTexture, texcoord);
        float r = color[3];
        if (r>AlphaFuncValue)
        {
            fragColor.xyz = fragColor.xyz*(1.0-r)+color.xyz*r;
            fragColor.w += r;
        }
 
        if (fragColor.w<color.w)
        {
            fragColor = color;
        }
        texcoord += deltaTexCoord; 
        --num_iterations;
    }
 
    if (fragColor.w>1.0) fragColor.w = 1.0; 
    if (fragColor.w<0.0) discard;
    gl_FragColor = fragColor;
}

[Dark Photon]

Please use [ code ] ... [ /code ] (without the spaces) around your code blocks. This keeps the indentation and avoids thread bloat. I've inserted them for you.

First thing, I notice you are doing a texture lookup using a texcoord value computed inside a conditional. That's something you need to watch out for as it can cause problems.

The specific texture lookup call you are using automatically computes the LOD from texture coordinate derivatives (i.e. texcoord deltas between adjacent fragments), and in the case of texcoords computed inside conditionals, sometimes adjacent pixels have no valid value for this texcoord and the derivatives are undefined (e.g. are possibly just flat huge).

Now you'd think that if you are not using MIPmapping, the LOD shouldn't make any difference, right? I fell into that trap as well. It's also used for anisotropic texture filtering too.

So first thing, make sure you've got GL_TEXTURE_MAX_ANISOTROPY_EXT nailed to 1 on that texture.

Second, I would stop using texture3D and use texture3DLod instead with a LOD of 0.