View Full Version : Visibility testing woes

Guillermo Amaral
11-04-2011, 12:33 AM
Hi guys, I've been trying to find out how to do some simple visibility testing/culling for a tilemap I'm working on, I've been searching for over a month with no success - I keep finding example involving 3D stuff and such but I'm working in 2D with glortho.

This is what I'm trying to do (pardon me if I don't use the correct terminology).

I'm trying to find the top-left and bottom-right visible coordinates in object coordinates (or what ever you call them) in order to perform some basic visibility testing.

I have been doing the math manually, but it doesn't take rotation into account for example.

Any help will be greatly appreciated. :)

11-04-2011, 02:01 AM
Do you have anu images you could post of what you are trying to achieve?
If it's just a regular 2D grid then I don't see the problem since it's a fixed size? ... and because of such, you know what you have to draw to each cell therefore it's easy to only draw the 'current cells' within the coordinate range of the window/camera.

Guillermo Amaral
11-04-2011, 07:25 AM
I want to use this not just for a simple tilemap, that's true, let's say I have this.

glOrtho(-40, 40, -30, 30, -1.f, 1.f);

So my visible range now is:

topleft( -40, 30 )
bottomright(40, -30)

I easily test visibility while rendering my tilemap. But what happens if I move around and more importantly *rotate* my camera.

* Camera moves to X position, Y zoom and Z rotation, let's say it's following something like a sprite on screen or something.

glRotatef(camera.rotation(), .0f, .0f, 1.f);
glScalef(camera.scale().x(), camera.scale().y(), 0.f);
glTranslatef(camera.translation().x() * -1, camera.translation().y() * -1, 0.f);

How do I get my topleft and bottomright now?

I'm pretty sure I have to do some matrix magic, but what.

I want to use this for not just tilemaps obviously, so I'm not really looking for a workaround, I need to figure out how to get the visible rect (if you will).

If it's still not clear I can post some pictures. :)

Guillermo Amaral
11-04-2011, 08:25 AM
Mmm I might have just figured it out, after some playing around with my calculator. Tell me if this makes sense.

Please bare with me, I might not use the proper nomenclature for most of this stuff.

I multiplied the MODELVIEW and the PROJECTION matrices, I then use that (as I would if I wanted to test visibility using NDC) - but instead of multiplying the resulting matrix against object coordinates in order to get NDC (-1..1) - I invert the resulting matrix and multiply NDC coordinates instead (-1.0, 1.0) to get the visible object coordinates.

Does it make sense to you more experienced folk?

If it does, I guess I would just need to offset the resulting vector using the camera position.