Porting rendering logic of GUI-toolkit to OpenGL

Hi everyone.

Introduction:
I am currently trying to port the renderer of a GUI-toolkit to OpenGL. It was originally implemented with AWT (abstract windowing toolkit). A naive implementation would be rather simple. I am quite certain however that the naive implementation would have horrible performance. This is why I ask here for some advice.

Problem:

[ul]
[li]A GUI consists of several Components. Rendering a Component means drawing a combination of potentially several colored shapes (triangles, circles, quads, lines) and pictures. [/li] [li]When rendering a Component the Components axis alinged bounding box must be taken into account. A Component is not allowed to render outside of its bounding box. [/li] [li]At any point in time the rendering logic for individual Components may change (for example when a button is pressed down).[/li][/ul]

Approach:

[ul]
[li]Circles, quads and lines can be represented as triangles. => all shapes are represented as triangles.[/li] [li]For text rendering I will use STB.[/li] [li]For the Bounding Boxes I will use glScissorTest[/li][/ul]

Naive Implementation:

[ul]
[li]For each Component (we assume they are already in correct order)[/li] [LIST]
[li]Do glScissor to set the scissor area (Bounding Box).[/li] [li]Upload as many untextured triangles as possible to a buffer and then draw it.[/li] [li]Upload as many textured triangles (pictures) as possible to a buffer and then draw it.[/li] [li]Continue with uploading textured and untextured triangle batches until everything was drawn. (We can not know in advance how many these will be!)[/li] [/ul]
[li]Alternative: [/li] [ul]
[li]When a component needs to be redrawn render the entire GUI to a texture.[/li] [li]While nothing has changed render the texture to screen.[/li] [/ul]
[/LIST]

Possible Problems:

[ul]
[li]Several draw calls per component could be quite costly![/li] [li]Batching must be done as an online-algorithm since we can not know how many shapes need to be drawn for the component.[/li] [li]The glScissor calls make batching multiple components together impossible.[/li] [li]An application that uses the GUI-toolkit will need to reset its own OpenGL state after each render pass.[/li][/ul]

I appreciate all advice on this. If there are any questions left unanswered please ask.
Thanks in advance.

This topic is cross-posted at: http://forum.lwjgl.org/index.php?topic=6148.msg32853#msg32853

Obviously the main thing is to get rid of the scissor test, as that will prevent batching.

One possible approach is to give each vertex an integer attribute containing the component ID. This would be used to retrieve the component’s bounding box from a uniform array, which would then be used to calculate gl_ClipDistance values. Alternatively, the bounding box could be passed to the fragment shader as a “flat”-qualified variable and the fragment shader would “discard” fragments outside of the bounding box.

If you want to isolate the GUI’s use of OpenGL from the application’s, use separate contexts.

[QUOTE=GClements;1281562]Obviously the main thing is to get rid of the scissor test, as that will prevent batching.

One possible approach is to give each vertex an integer attribute containing the component ID. This would be used to retrieve the component’s bounding box from a uniform array, which would then be used to calculate gl_ClipDistance values. Alternatively, the bounding box could be passed to the fragment shader as a “flat”-qualified variable and the fragment shader would “discard” fragments outside of the bounding box.

If you want to isolate the GUI’s use of OpenGL from the application’s, use separate contexts.[/QUOTE]

So what you are saying is that I should do the scissor test myself in the fragment shader, correct? With a conditional branch or is there some kind of build in functionality for that?
If I could do this it would help tremendously with batching of shapes. But since the number of shapes per component can vary wildly in between frames I would still have to fill the buffer per frame which is not a good idea, right?
Would you say it is a good idea to render to texture instead? Then I would only have to re-render dirty parts of the texture each frame.

Thanks so far!

Either in the vertex shader (via user-defined clip planes) or the fragment shader. I’m not sure which one would be more efficient in practice (if the contents are much larger than the bounding box, using the vertex shader will eventually win).

If the rendered primitives can include wide lines or points, you need to perform a scissor test in the fragment shader even if you clip the geometry in the vertex shader.

The fragment shader can discard fragments with the “discard” statement. The result is as if the fragment hadn’t been generated (i.e. the colour, depth and stencil buffers will be unchanged for that fragment).

The data being rendered shouldn’t change completely every frame. If you know what data has changed and what hasn’t (and if you don’t know, that suggests an API deficiency), you should be able to update only the data which has changed.

That can help, but it doesn’t necessarily eliminate the need to improve rendering efficiency. If you’re caching an entire window (or other “container” widget) as a single texture, operations such as scrolling require redrawing everything.

It’s hard to be specific given that the details depend heavily upon the capabilities of the GUI library, e.g. whether widgets can overlap, whether you need to support smooth animation (e.g. fading), etc.