I would like to test for the (partial) visibility of a rectangle in model space, when rendered into viewport space. I'm currently using Gl's GL_SELECT mode, but there are speed issues with this.

Problem reduces to: an algorithm to test for overlap between a convex quadrilateral Q (the projection into viewport space) and a rectangle R (the viewport itself).

If any vertex of Q is inside R, there's overlap (trivial).

If any edge of Q crosses an edge of R, there's overlap (Cohen-Sutherland, perhaps?)

Finally, if Q encloses R, there's overlap (can't see any easy way to do this).

Can anyone help?