First of all you need a compiler ;o). A good and free one can be found at www.bloodshed.net. OpenGL libraries are included I think. Then you should read The Red Book, which is also freely available all over the web.
The Red book shows you how to use the Opengl function calls while the Blue book is a reference for the functions. If you have the bucks, get the Opengl Superbible. It has some good stuff in it.
Well, if you’re going to be compiling for those two pretty disparate platforms, I would recommend gcc as your compiler (it works on nearly every platform in existence). http://gcc.gnu.org/ It is Free Software. The Cygwin port of GCC to Windows NT includes OpenGL stuff (I’m 90% sure of that). See http://www.cygwin.org/