How to antialias 2D polygons

OpenGL OpenGL: Basic Coding
1

Hello all,

I’m trying to use the 2d polygon tesselation method using Stencil buffer as described in Red Book: http://www.glprogramming.com/red/chapter14.html#name13

I can get the correctly tesselated polygons this way. But the problem is how to make the resulted shape antialiased?
I know I could use FSAA but this is incompatible=not a good option for me.

Are there any other ways how to antialias outlines of the resulted stencil buffer mask?

2

Why is FSAA incomptabile=not a good option?

There are other ways to get some anti-aliasing without using FSAA, but it’s a good idea to specify why FSAA won’t work so people don’t recommend other methods that might also not work for whatever reason FSAA won’t.

One method to get some anti-aliasing is to blur the image at boundaries where you think there’s a separation between objects. You can guesstimate this using differences in depth and color, which is the defacto way I believe in current deferred rendering methods.

Stalker used a edge detection algorithm to determine which areas could be blured to get rid of some ugly jaggies.
http://http.developer.nvidia.com/GPUGems2/gpugems2_chapter09.html

3

strattonbrazil,

“why FSAA won’t work”

  1. It is not supported on all gfx HW. Especially lot of notebooks don’t have the FSAA extensions etc.
  2. Even if it is supported by the HW usually it cannot be turned on/off once the display is initialized. I need that the AA mode can be controlled by the user during rendering phase.
  3. I need to antialias only parts of the result. For example smaller fonts should be aliased.

Hope this will clarify my needs.

Also I don’t want to use Shaders as not all HW can do it.

I’d prefer some solution which can be combined with the ‘tesselation using Stencil buffer’ technique described in Red Book.

I should also note that I tried to use the built-in Tesselator which is present in GLU library but the tesselator slows down too much and there is also problem how to antialias the resulted triangulated shapes only on edges and not ‘inside’ the complex shape.

I appreciate any ideas. Thanks!

4

Even if it is supported by the HW usually it cannot be turned on/off once the display is initialized

glEnable/Disable(GL_MULTISAMPLE). Simple. You still have the multisample buffers, but they effectively work like single-sample buffers.

5

Well, IMHO that’s the sweet theory. I have read that lot of GFX cards doesn’t react on this flag once the multisampling is on so it looks like yet another classic ‘messy implementation’ case.