1. glMatrixMode( GL_MODELVIEW )
glLoadIdentity()
2. Set New Camera Pos/Rot
Shouldn't you be setting the camera orientation with GL_PERSPECTIVE, not GL_MODELVIEW? Which is our point...the modelview matrix has nothing to do with the camera position.