Fill Polygons according to winding number rules

OpenGL OpenGL: Basic Coding
1

Hello everybody. Im new to OpenGL. As a project im assigned to create a simple program that creates polygons.
When you click on the window a vertex appears and with each subsequent click the vertexes connect with each other which results in creating a polygon.
Im assigned to fill the polygons according to some rules. Two of them are the “non zero” winding rule and the “odd” winding rule.
Ive the done the part where you create the polygons. But i struggle to find helpful info for the “gluTess” functions and simple examples.
Does anybody have any idea where to start from? I’ve read some tutorials on glprogramming site and Khronos site but these doesnt help a lot a beginner.
Thanks in advance!

Edit: Lets say for example i want to fill this polygon. What is going wrong with this code? Again im sorry if this example is totally wrong but im really trying to find out how the “glutess” works

#include <windows.h>
#include <stdlib.h>
#include <stdio.h>
#include <GL\glut.h>
GLdouble rect[4][3] = {{50.0, 50.0, 0.0},
	                          {200.0, 50.0, 0.0},
	                          {200.0, 200.0, 0.0},
	                          {50.0, 200.0, 0.0}};


void beginCallback(GLenum which){
   glBegin(which);
}
void endCallback(void){
   glEnd();
}
void errorCallback(GLenum errorCode){
   const GLubyte *estring;

   estring = gluErrorString(errorCode);
   fprintf (stderr, "Tessellation Error: %s
", estring);
   exit (0);
}


void display(){

	glColor3f(0, 0, 0);
	glPointSize(5);   
	glMatrixMode(GL_PROJECTION);
	glLoadIdentity();
	gluOrtho2D(0.0, 500.0, 0.0, 500.0);

	GLUtesselator *tobj = gluNewTess();
	gluTessCallback(tobj, GLU_TESS_ERROR, (void (__stdcall*)(void))glVertex3dv);
        gluTessCallback(tobj, GLU_TESS_ERROR, (void (__stdcall*)(void))endCallback);
	gluTessCallback(tobj, GLU_TESS_ERROR, (void (__stdcall*)(void))beginCallback);
	gluTessCallback(tobj, GLU_TESS_ERROR, (void (__stdcall*)(void))errorCallback);


	   glShadeModel(GL_FLAT);
	   gluTessBeginPolygon(tobj, NULL);
	      gluTessBeginContour(tobj);
	      glColor3f(1, 0, 0);
	         gluTessVertex(tobj, rect[0], rect[0]);
	         gluTessVertex(tobj, rect[1], rect[1]);
	         gluTessVertex(tobj, rect[2], rect[2]);
	         gluTessVertex(tobj, rect[3], rect[3]);
	      gluTessEndContour(tobj);

        
          glFlush();

}

int main(int argc, char** argv)
{
   glutInit(&argc, argv);
   glutInitDisplayMode(GLUT_RGB | GLUT_SINGLE);
   glutInitWindowSize(500, 500);
   glutInitWindowPosition(100, 100);
   glutCreateWindow(argv[0]);
   glutDisplayFunc(display);
   glutMainLoop();
   return 0;
}
2

The easy way to do this is to draw the polygon as a triangle fan using stencil operations. The initial point should be somewhere outside of the polygon (it doesn’t have to be on screen).

If you can rely upon OpenGL 2.0 or later, you can use glStencilOpSeparate() to select separate operations for front and back faces (increment for front, decrement for back). Otherwise you need to draw the fan twice, each time drawing only front faces or only back faces.

After rendering the fan, the value in the stencil buffer will be the winding number. You can fill the appropriate portion by rendering a screen-sized quad with stencil testing enabled and the stencil function set according to which winding rule you wish to use.

For even-odd winding, you don’t need to distinguish front and back faces; you can just use GL_INVERT as the operation (and you only need a 1-bit stencil buffer).

It’s been too long since I used the GLU tesselator for me to give any advice on that approach.

3

Here’s a really old FAQ that has a section on the GLU tessellator routines. Search for “concave” in here: LINK.

As for the concave polygon rasterization algorithm GClements mentioned, here’s a short write-up on that: Drawing Filled, Concave Polygons Using the Stencil Buffer. Yes , do keep in mind with modern OpenGL you don’t need multiple stencil passes for this anymore.

4

Thanks for the replies guys. After a long night it seems that i finally made it. Since now it works with the 2 rules (winding odd, and winding nonzero) using the gluTessProperty(tess, GLU_TESS_WINDING_RULE,windingRule) where windingRule=GLU_TESS_WINDING_ODD or GLU_TESS_WINDING_NONZERO.

The thing is that now i have to fill the polygons with another set of rules. Third rule is to fill every polygon no matter what the winding number is and 4th rule is to fill only the polygons that have (windingNumber/2)=ODD. (for example 3/2=1.5 , you keep only the 1.0 which is ODD so you fill that polygon). Any help on this one by using the gluTess functions? Thanks again!