virtual object occluded by real object

[deduu10]

hi guys, does anyone interest in Augmented Reality? I'm trying to occlude my virtual object created by openGL with the real one. what I'm trying to achieve is like this one http://www.youtube.com/watch?v=dbpaLdVEE0g

I'm pretty sure that by using the depth buffer I can occlude the virtual with the real one. But after conducting numerous trials, I havent succeed. Does anyone want to help me to figure out?


thanks

[aqnuep]

How you plan to acquire the depth buffer of the real scene? This is a non-trivial problem that most of the times requires multiple cameras to reproject the images captured by the cameras into 3D space.

In the video you've linked probably doesn't perform a true reprojection, but simply handles this special scene, i.e. uses the knowledge that the occluder object is circle shaped and uses a hardcoded value to define it's distance (or recalculate the distance based on knowing the size of the object). That could be easily done by doing a Hough transform to determine the position and size of the circle representing the occluder, however, in general case such an approach won't work.

[ZbuffeR]

Provided you know the position/shape of the real occluder, it is only a matter of render video background, render virtual occluder with a depth buffer and color mask so that color buffer is not updated, then virtual objects with depth testing normally.

[deduu10]

thanks for replying.

zbuffer, can you write the pseudocode for obvious explanation?

thanks

[Kopelrativ]

I am curious, how do you get the positions of the real world objects?

[aqnuep]

thanks for replying.

zbuffer, can you write the pseudocode for obvious explanation?

thanks

glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
renderOccluders();
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
renderOccludees();

This is what zbuffer explained and that's the only OpenGL related code needed to implement what he proposed. You have to figure out the rest.

[deduu10]

glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
renderOccluders();
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
renderOccludees();

This is what zbuffer explained and that's the only OpenGL related code needed to implement what he proposed. You have to figure out the rest.

kopelrativ, I use marker to get the position of the real occluder.

aqnuep, thanks for writing the code. I have ever tried that code before, my virtual occluder succeed occlude my virtual object, but when I used colormask, indeed my virtual occluder was not rendered, but it was occluded by my virtual object. I will keep trying another way. Thanks

[aqnuep]

aqnuep, thanks for writing the code. I have ever tried that code before, my virtual occluder succeed occlude my virtual object, but when I used colormask, indeed my virtual occluder was not rendered, but it was occluded by my virtual object. I will keep trying another way. Thanks

Well, then you've probably done something wrong. Let me rephrase then the explanation:

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
 
glDisable(GL_DEPTH_TEST);
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
 
render real world picture as a full screen quad
 
glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
 
render simple geometry (e.g. sphere) that acts as the virtual occluder
 
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
 
render your virtual objects (whatever it be) that acts as the virtual occludee

If you follow this method, it has to work, otherwise there may be a different problem with your code.

[deduu10]

Well, then you've probably done something wrong. Let me rephrase then the explanation:

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
 
glDisable(GL_DEPTH_TEST);
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
 
render real world picture as a full screen quad
 
glEnable(GL_DEPTH_TEST);
glDepthMask(GL_TRUE);
glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
 
render simple geometry (e.g. sphere) that acts as the virtual occluder
 
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
 
render your virtual objects (whatever it be) that acts as the virtual occludee

If you follow this method, it has to work, otherwise there may be a different problem with your code.

thanks for your explanation, I had tried this one also.. the problem is my real scene was cleared to black as I put GL_COLOR_BUFFER_BIT to glClear(). These code is supposed to work if I know how to maintain my real scene..

[aqnuep]

thanks for your explanation, I had tried this one also.. the problem is my real scene was cleared to black as I put GL_COLOR_BUFFER_BIT to glClear(). These code is supposed to work if I know how to maintain my real scene..

Presenting your real scene goes to the line I've noted as "render real world picture as a full screen quad", not before glClear. That way it cannot clear your image.