OpenGL tessellation, problem tessellating concaves

[jacobholm]

I send in the vertices to tessellator (see image) using gluTessVertex().

http://img199.imageshack.us/img199/22/tessellation.jpg

As you can see it's a "quad" with an indent. I have registered my own GLU_TESS_VERTEX callback, tessVertexCB(). If I understand correctly this function will be called for each vertex, in the order of each triangle, i.e. the three first calls to this function will give me the first triangle, the next three calls will give me the second triangle etc.. Am I wrong?

These are the triangles I am getting (by index as in image):

1st. 1, 4, 5
2nd. 4, 1, 3

Already here the second triangle is wrong, i was expecting 2 as the last point in the second triangle. If vertex nr 4 is farther to the right the triangle 4, 1, 3 will intersect the 5, 4 line. Have I misunderstood how this works?

Regards,
Jacob

[mfort]

I do not see any problem here. The tessellator did the job correctly. The second triangle is not wrong.

If you change the input (position of vertex #4) then you obviously get different result.

[Ondras]

The tessellation output is not clear for me either but I guess registering the GLU_TESS_VERTEX callback is not enough. As far as I remember the GLU tessellator provides optimal sequence of primitives. The primitives might be triangles but also triangle fans or strips. In order to track the the primitive type you must also register GLU_TESS_BEGIN and GLU_TESS_END callbacks.

[jacobholm]

http://img6.imageshack.us/img6/22/tessellation.jpg

The triangle marked in red(4, 1, 3) is one of the triangles I am getting. I will try to use GLU_TESS_BEGIN and END, thanks.

[jacobholm]

I just checked, and they are GL_TRIANGLES in GLU_TESS_BEGIN .. =(

I have not implemented the combine callback, could this have something to do with it?

[jacobholm]

I found the problem. I was sending in 2D points, converting them back to 3D after the tessellation. The only problem was that for each point I sent in pointers to a list with 2D points, causing the Z value to become the X value of the next point.

*facepalm*

Hopefully some other newbie will read this and not have to post...