I am working on a C++ app on Intel MAC OSX 10.6.x. I have a variable which contains pixel data which was obtained using OpenGL command glReadPixels. I want to do some operations directly on the pixel data using x86_64 assembly instructions. The assembly routine works fine in test programs but when I try to use it on the pixel data, it only gets zeroes in the memory location pointed by the pixel data variable. I am guessing this is since I am trying to access video memory directly from x86_64 assembly. Is there a way to access x86_64 OpenGL video or graphics memory directly from assembly? Otherwise how can I resolve this situation?
Have you verified that glReadPixels() is filling your data array with valid data? (by looping through the array and doing a printf()) It may be that glReadPixels() is returning black, or no pixels at all. Try memset’ing the preallocated data array to some non-zero value like 0xAD, before your glReadPixels() call to see if you are actually reading any data back at all.
If you want to work with video memory directly, you’d have to use OpenCL, which OSX 10.6 has native support for. But you’re out of luck if you want to use x86.