PDA

View Full Version : Object picking stencil buffer issue



Keithster
01-01-2016, 03:52 PM
Greetings!

I'm using the stencil buffer to achieve easy object picking. The problem is, when I render my UI layer I can still click a button and if the button was in front of an object I rendered previously, it would go through and still select that object. I want the UI to block objects selections in the scene.

(See Image below)

Here's how I'm using the stencil:



glClearStencil(0);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
glEnable(GL_STENCIL_TEST);

ARRAY_FOREACH(entity *it, Game->Entities)
{
glStencilFunc(GL_ALWAYS, i + 1, -1);
// render entity
}



And then in my editor code I query the stencil value and do object selection based on that:



u32 StencilIndex;
glReadPixels(Mouse->X, GAME_HEIGHT - Mouse->Y - 1, 1, 1, GL_STENCIL_INDEX, GL_UNSIGNED_INT, &StencilIndex);
Log("Stencil %d", StencilIndex);
ForCount(EntityCount)
{
entity *it = Entities[i];
b32 Selected = Mouse->Left == Key_Down && i == StencilIndex - 1;
if (Selected)
{
Editor->Selection = it;
Editor->TransformType = Transform_None;
break;
}
}


Then I render the UI, I tried setting the stencil value to 0 but it never sets it, it always keeps the last value that was in the buffer:



glStencilFunc(GL_ALWAYS, 0, -1);
// ... render UI


Image: http://i.imgur.com/yAH0v1t.png (It should print 0)

What am I doing wrong?

Any help is appreciated.

Thanks!
-Keithster