I would also like to do this. I had some dificulty selecting vertices though. I had to change these vertices to crosses formed with lines and then it worked fine. If you find out how do this I would apreciate if you could inform me about this. Thanks.
P.S. Maybe you should post this on the advanced forum.
If you want to draw a rectangle on the screen and get all vertices inside this rectangle, do the following:
You got the coordinates x1/y1 and x2/y2 of the rectangle selection. Then you do a gluProject for each 3D-vertex you want to check. The result is the 2D-point on screen for each vertex. Then simply check if it’s inside the rectangle.
Is it possible to do this with the select buffer by taking the coordinates at the center of the box and creating a picking matrix for the whole rectangle instead of just a few pixels around the cursor?