Tessellation crashes in Win7 but not linux

OpenGL OpenGL: Advanced Coding
1

I’m drawing an enclosed concave polygon (filled with a color) using the GLUtesselator. The opengl calls that do this are generic. The problem is that it works fine on linux, but the call to gluTessEndPolygon() causes a crash in windows 7 (access violation). I do use these functions:

GLU_TESS_BEGIN
GLU_TESS_END
GLU_TESS_ERROR
GLU_TESS_VERTEX
GLU_TESS_COMBINE

There is no crash inside of these functions. It happens after the call to glu_tess_end(). glu_tess_error() is not called, and glu_tess_combine() is called once.

Does anyone have any idea about what could cause this or how to debug it?

thanks,
William

2

Not to be glib but have you tried using a debugger? Lots of unexpected crashes can result from seemingly benign coding shenanigans (voice of experience).

3

I’m using Visual Studio and have stepped through every line of my code. I also step through the linux version with gdb / netbeans (no crash). I know the line of code that it crashes on and that is inside opengl.

William

4

Check your calling conventions (cdecl, stdcall, …)

You have to declare your functions like this:
void CALLBACK myFuncCallback(…);

If not, the app crashes. The CALLBACK macro is equivalent to __stdcall. Default calling convention in MSVC is cdecl.

Linux does not have this mess so it works fine without it.

5

Here’s the way they are specified. For example:

gluTessCallback(tess, GLU_TESS_BEGIN, ( GLvoid(__stdcall *)() ) &glu_tess_begin);

I hit breakpoints inside all of these function except GLU_TESS_ERROR. Inside the GLU_TESS_END function I just call glEnd(). This is the function that crashes.

William

6

I found it. One of the prototypes and its function WAS missing the CALLBACK term.

thanks,
William