PDA

View Full Version : Calculating all pixels in a frame



genzm19
05-11-2013, 02:55 PM
Hy,
I didn't know if this is the right forum or the right category to post this thread, so please move this thread if necessary, or point me to a more relevant forum.
Anyway, here is my question:

I was doeing some research on computer graphics. I want to build a system that can render frames and display them. Herefor I made an 2D array which acts as a buffer. This way I can fill up all the pixels and afterwards display the right color data.

But here is my problem: Filling the buffer requires a double for-loop over 800x600 pixels. And this is takes an enormous amount of time. This is actually not that strange since we're speaking of 480000 pixels. But that's when I started wondering how this is done in current technologies. Because somehow a game, or application, even the OS is capable of refreshing the window at a framerate of 60Hz. On a fullHD screen this means calculating 124.416.000 pixels in one second, or about 8 nanoseconds per pixel. This seams to me as an enormouse speed, as this about the speed of a modern processor.

I've been thinking about it and I can't seem to figure it out, my possible explenations:

GPU parallel programming:

As a GPU can execute many tasks at the same time, each pixel could be calculated in a thread. This would give an enourmous speed boost. But computers didn't have always GPU's and parallel programming is quite new. So how was this done before this?

partial refresh:

I somehow the computer could predict which pixels need refresh and which can stay the same, we could limit calculations to the new pixel values. This would help, but it's clear that this is not the solution. A movie sequence with new pixels every frame can be displayed perfectly.

pixel calculations:

If the calculations for the color of a pixel could be simplified, it would increase the speed of the entire process significantly. But in my program, I had a method called pixel_calculation. And in my current implementation it just returns a constant value. So no calculations are done, and this seems to me the fastest you can get, and yet still it is incredibly slow.

These were the options I could come up with, but at the same time, I could easily counter them. I've been struggeling with this for a long time now. Can anyone explain to me how the color value of every pixel is calculated in such a high speed?

Thanks for the help

Alfonse Reinheart
05-11-2013, 09:50 PM
But computers didn't have always GPU's and parallel programming is quite new. So how was this done before this?

Even older computers had dedicated video hardware. They usually had some basic 2D acceleration. In the DOS days, the text console was really just pulling glyphs from image data and throwing them to the screen. That's not hard to do quickly; the old Apple IIs and so forth could pull that off.

Later, you started to get some 2D graphics acceleration with simple 2D primitives: lines and boxes. Also, you started to get memory blitting: fast-block copies of data from one location in video memory to another. Such an operation is fast because the graphics processor can initiate a direct memory transfer from one location of memory to another. This DMA copy goes as fast as the memory will allow it to go.

You generally do not "compute each pixel", ever. You start with a blank screen and draw stuff on top of it via a series of primitive drawing calls.

Remember: drawing calls are just fancy memory setting operations for local graphics memory. Most of the performance gained in such scenarios comes from the fact that you're not using cached, CPU memory. That the graphics processor can access its local memory faster than the CPU can access main memory. The graphics processor's memory architecture is designed specifically to make these operations fast.

Oh, and don't forget: older computers did not render at "fullHD" resolutions at all. The best older computers could do was 800x600 or maybe 1024x768.


But in my program, I had a method called pixel_calculation. And in my current implementation it just returns a constant value. So no calculations are done, and this seems to me the fastest you can get, and yet still it is incredibly slow.

No, that's not "the fastest you can get." The "fastest" you can get using pure CPU code involves understanding things about memory architecture, bypassing the CPU cache (depending on the circumstances), DMAs or write-combining, and other stuff you don't get taught in Introduction to C++.

Simply looping over a block of memory and writing a constant value is not the fastest way to set a block of memory to that value.

genzm19
05-12-2013, 02:49 AM
Ok, I think I understand that. Just a few more things that came in my mind:

So how does OpenGL work with this? Because as far as I know, OpenGL does not only update the pixels that changed. When calling glClear(), everything is erased. Than the entire frame is rebuild with the new data. This would mean, that somewhere in the OpenGL pipeline, the color of every pixel is calculated (I suppose this is done in the rasterisation phase?). Could you please explain to me how this is done?

Second, I assumed that cache memory is quite fast. So graphics memory is even faster? How fast is it than, and how come so that it's so much faster?

I know full HD is a little bit much, but I'm not even able to render 800x600 on an intel i7 2nd generation. So it's clear that this is definitly not the way it is done.

And I know this is no basic C++. Still I'd like to get more into this. I really wonder how I could achieve this myself? Is it even possible, to write a program in C or C++ that can acces these DMAs, of graphics memory. Would you like to give me more explenation, or point me to sites or books that can help me understand how this is done?

genzm19
05-12-2013, 03:32 AM
OK, I just read about OpenGL's pixel buffer objects. Is this an interessting subject?
If I would store my pixel data in a PBO, and than somehow display this, would this be faster?

Alfonse Reinheart
05-12-2013, 04:30 AM
So how does OpenGL work with this? Because as far as I know, OpenGL does not only update the pixels that changed. When calling glClear(), everything is erased. Than the entire frame is rebuild with the new data. This would mean, that somewhere in the OpenGL pipeline, the color of every pixel is calculated (I suppose this is done in the rasterisation phase?). Could you please explain to me how this is done?

You seem to be looking at this backwards. You're looking at a pixel and trying to work backwards to figure out how it got its color.

OpenGL executes rendering commands. These commands set pixels to a color. Other, later, rendering commands may set those pixels to different colors. Sometimes, it may combine a color with the current pixels color.

If I tell OpenGL to draw a square, it draws a square. The system figures out which pixels need to have their color changed to the color of the square, and it changes those pixels to that color. Same goes for drawing a line, triangle, point, or anything else.

Deferred rendering can work backwards (but only by "cheating"). But you're not ready for that.


I know full HD is a little bit much, but I'm not even able to render 800x600 on an intel i7 2nd generation.

Render to what? Your code should look no more complex than this:



unsigned int GetColor() {return 0xFF0000FF;}

std::vector<unsigned int> colors(800*600);

for(int x = 0; x < 800; ++x)
{
for(int y = 0; y < 600; ++y)
{
colors[x + y * 800] = GetColor();
}
}

//Rendering complete


It is far more likely that your performance issue is with the display of this image, not in computing it.


Is it even possible, to write a program in C or C++ that can acces these DMAs, of graphics memory.

No. Only the OS can touch graphics memory directly, because it is a resource that is shared among all programs in the system.

genzm19
05-12-2013, 06:18 AM
Well, I do think I'm ready for that. I think you underestimate me.
Here is my target: I've figured out a way to calculate geometry and objects very easy and very fast. So I can get the data for a certain pixel with a very easy algorithm, I just need to find a way to display this.
There's got to be a way? If OpenGL can set pixel colors, and OpenGL is also just a piece of software running on top of the OS, than my software should be able to do that as well.

At the beginning of OpenGL, someone must have written code, to set color pixel data (probably in C or C++, I'm not sure). But he did it, so I can do it again? I know that what I ask isn't an easy programming question.
I might just be rewriting OpenGL, and I know that that is actually quite advanced stuff, but still, it can be done.

Alfonse Reinheart
05-12-2013, 10:37 AM
Here is my target: I've figured out a way to calculate geometry and objects very easy and very fast. So I can get the data for a certain pixel with a very easy algorithm, I just need to find a way to display this.

So your actual question is "how do I display an image quickly?" Why didn't you just say so? Why all the misleading stuff about "calculating pixels" and such?

Also, I highly doubt your "very easy and very fast" purely CPU method would hold a candle to the actual GPU, but then again, you never said what exactly you were doing.


If OpenGL can set pixel colors, and OpenGL is also just a piece of software running on top of the OS, than my software should be able to do that as well.

OpenGL is not "just a piece of software running on top of the OS". It is not a library that you download. It is implemented as part of the graphics drivers. Therefore, a portion of the OpenGL implementation is, effectively, part of the OS. Indeed, Apple even embeds OpenGL into the OS itself for MacOSX.

Your software is not part of the OS. You therefore do not have direct access to the graphics card. The only thing you can do is what OpenGL, D3D, or some other rendering API allows you to.

The fastest you're going to get is to build your image data, transfer it to a buffer object (http://www.opengl.org/wiki/Buffer_Object) via some form of streaming technique (http://www.opengl.org/wiki/Buffer_Object_Streaming) (building it directly into mapped buffer object data will likely be terrible for performance), use the buffer as a pixel buffer (http://www.opengl.org/wiki/Pixel_Buffer_Object) to (asynchronously) upload that data to a texture, then blit from that texture (http://www.opengl.org/wiki/Framebuffer#Blitting) (technically, a framebuffer holding that texture) into the backbuffer. And then do a buffer swap. You probably want to double-buffer your textures (and thus have multiple FBOs to blit from).

genzm19
05-12-2013, 12:59 PM
OpenGL is not "just a piece of software running on top of the OS". It is not a library that you download. It is implemented as part of the graphics drivers. Therefore, a portion of the OpenGL implementation is, effectively, part of the OS. Indeed, Apple even embeds OpenGL into the OS itself for MacOSX.

So just to be clear, what you're saying is that all speed is coming from the fact that OpenGL works on the graphics card?


Also, I highly doubt your "very easy and very fast" purely CPU method would hold a candle to the actual GPU, but then again, you never said what exactly you were doing.

As my project is still in a very early development stage, I've implemented on the CPU just to make it easier to work with, debug, improve,... Off course, if everything goes right and my algorithm works, I will definitly port it to the GPU (probably via CUDA or OpenCL). So yes, for now it will be just on the CPU.

and finally: sorry, that I haven't been more clear about what I wanted. I will be looking into buffer objects more now, but I think this is what I really need. Anyway, thanks for the help and information. I hope I will find my answers in the links you showed me. If not, I will post my new questions here.

genzm19
05-13-2013, 01:20 PM
Ok, I'm kind of stuck on a few parts...

First, how is color data formatted in OpenGL? I've read this, http://www.opengl.org/wiki/Image_Format. But this doesn't really help. It's just so I can know how to fill my buffer.
Second, how is blitting done to the backbuffer? I assumed that the backbuffer is bound to GL_WRITE_FRAMEBUFFER, but I'm not sure.
Third, how is the swapping done? I couldn't find any openGL call that can swap front and back buffer.

So here is what I got so far:



// WIDTH and HEIGHT were defined as 800 and 600
GLuint BUFFER;
glGenBuffers(1,&BUFFER);
glBindBuffer(GL_READ_FRAMEBUFFER,BUFFER);
glBufferData(GL_PIXEL_UNPACK_BUFFER,WIDTH*HEIGHT,N ULL,GL_DRAW_BUFFER);
unsigned char* buffer_map = (unsigned char*) glMapBufferRange(GL_PIXEL_UNPACK_BUFFER,0,WIDTH*HE IGHT,GL_MAP_WRITE_BIT);

bool running = true;
while (running) {
if (isFinished()) {
running = false;
}

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

fill_buffer(buffer_map, WIDTH, HEIGHT,objects);
glBlitFramebuffer(0,0,WIDTH,HEIGHT,0,0,WIDTH,HEIGH T,GL_COLOR_BUFFER_BIT,GL_FILTER);
}

void fill_buffer(unsigned char* buffer, unsigned int width, unsigned int height) {
for (unsigned int i = 0; i < width; ++i) {
for (unsigned int j = 0; j < height; ++j) {
if (calc_pixel(i,j)) {
// somehow fill the buffer with red (or any other color) using
buffer[i*width+j];
}
}
}
}

Alfonse Reinheart
05-13-2013, 04:07 PM
I've read this, http://www.opengl.org/wiki/Image_Format. But this doesn't really help.

Of course it doesn't. That describes the format of the data stored in OpenGL. What you want is how the data you pass to OpenGL should be formatted. It's essentially dealer's choice; you describe what your data looks like (http://www.opengl.org/wiki/Pixel_Transfer), and OpenGL will figure out how to make it work.


Second, how is blitting done to the backbuffer?

What do you mean by "how"? Are you asking what the commands you need to use are (http://www.opengl.org/wiki/Framebuffer#Blitting), or what the actual process is?


Third, how is the swapping done? I couldn't find any openGL call that can swap front and back buffer.

There isn't one. The disposition of the default framebuffer (http://www.opengl.org/wiki/Default_Framebuffer) is not something OpenGL specifies. You have to use platform-specific commands to do so. Or if you're using an appropriate abstraction layer, then you use that layer's functions.

genzm19
06-07-2013, 06:26 AM
Sorry, I've been away for a while. I've just started again. This is what I got so far:



int main() {
GLuint BUFFER;
glGenBuffers(1,&BUFFER);
glBindBuffer(GL_READ_FRAMEBUFFER,BUFFER);
glBufferData(GL_PIXEL_UNPACK_BUFFER,WIDTH*HEIGHT,N ULL,GL_DRAW_BUFFER);
unsigned char* buffer_map = (unsigned char*) glMapBufferRange(GL_PIXEL_UNPACK_BUFFER,0,WIDTH*HE IGHT,GL_MAP_WRITE_BIT);

bool running = true;
while (running) {
if (isFinished()) { running = fallse; }

glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);

fill_buffer(buffer_map, WIDTH, HEIGHT,objects);
glBlitFramebuffer(0,0,WIDTH,HEIGHT,0,0,WIDTH,HEIGH T,GL_COLOR_BUFFER_BIT,GL_FILTER);

window.display();
}

return 0;
}

void fill_buffer(unsigned char* buffer, unsigned int width, unsigned int height) {
for (unsigned int pixelX = 0; i < width; ++pixelX) {
for (unsigned int pixelY = 0; j < height; ++pixelY) {
buffer[pixelX*width+pixelY] = calc_pixel(i,j) ? 1 : 0; // writing a value of 1 to the buffer for test reason
}
}
}

// buffer only contains red values
glDrawPixels(width,height,GL_RED,GL_UNSIGNED_BYTE, buffer);
}

Note that this is only the essential portion of the code. I don't know if this looks good or if this might work?

I wanted to test it, but I couldn't compile it, becaus I got "unresolved external symbol" for some functions. I know what that means: that there is no implementation available for some of the functions I'm using.
I get this error with: glewBindBuffer, glewBufferData, glewGenBuffers, glewBlitFramebuffer, glewMapBufferRange
So at the moment I'm figuring out why my glew isn't properly working... Any suggestions are welcome, but off course it's a difficult for you to guess what might be wrong.
Any suggestions on the could would be greatly appreciated.

genzm19
06-10-2013, 07:49 AM
Ok, I solved the unresolved symbol problem. Stupid of me...

Anyway, I started the code, and I got an acces violation on the second line:


GLuint BUFFER;
glGenBuffers(1,&BUFFER);

I don't really see how I could have done something wrong in these lines, but apperantly I did. So can someone please point it out to me?

Alfonse Reinheart
06-10-2013, 08:05 AM
I don't see the part of your code where you initialize your OpenGL loading library (http://www.opengl.org/wiki/OpenGL_Loading_Library). Or OpenGL for that matter. You should do that before calling OpenGL functions.

genzm19
06-10-2013, 08:13 AM
I was just about to edit my post. so I didn't have any initation code...

Anyway, what I found on your (link but I just had found that myself):

glewExperimental=GL_TRUE;
GLenum err=glewInit();
if(err!=GLEW_OK)
{
//Problem: glewInit failed, something is seriously wrong.
cout<<"glewInit failed, aborting."<<endl;
}
So when I run this I get an error code from the glewInit() function. So I'm just looking around on google what possible reasons are.

edit: found it and my error says: "error: missing GL version"

genzm19
06-10-2013, 10:05 AM
So the "error: missing GL version" is fixed. I created my openGL context after the glewinit. didn't notice...
Anyway, to continue, this call:


unsigned char* buffer_map = (unsigned char*) glMapBufferRange(GL_PIXEL_UNPACK_BUFFER,0,WIDTH*HE IGHT,GL_MAP_WRITE_BIT);
returns a NULL object. so the mapping didn't work I assume?

edit:
I figured I just used wrong parameters? So I tried some different ones, but none work, this is the code right now:

glGenBuffers(1,&BUFFER);
glBindBuffer(GL_READ_FRAMEBUFFER,BUFFER);
glBufferData(GL_PIXEL_UNPACK_BUFFER,WIDTH*HEIGHT,N ULL,GL_DYNAMIC_DRAW);
unsigned char* buffer_map = (unsigned char*) glMapBuffer(GL_PIXEL_UNPACK_BUFFER, GL_WRITE_ONLY);

genzm19
06-11-2013, 05:05 AM
Just one more paramtere edit and it worked:

glGenBuffers(1,&BUFFER);
glBindBuffer(GL_PIXEL_UNPACK_BUFFER,BUFFER);
glBufferData(GL_PIXEL_UNPACK_BUFFER,WIDTH*HEIGHT,N ULL,GL_DYNAMIC_DRAW);
unsigned char* buffer_map = (unsigned char*) glMapBuffer(GL_PIXEL_UNPACK_BUFFER, GL_WRITE_ONLY);

Now I'm able to fill the buffer very fast. Much faster than I really expected.

Now just as a test case, I filled the buffer with all 1 values (being completely red).
This is the code:


for (unsigned int pixelX = 0; pixelX < width; ++pixelX) {
for (unsigned int pixelY = 0; pixelY < height; ++pixelY) {
buffer[pixelX*width+pixelY] = 1;
}
}

glDrawPixels(width,height,GL_RED,GL_UNSIGNED_BYTE, buffer);

glBlitFramebuffer(0,0,WIDTH,HEIGHT,0,0,WIDTH,HEIGH T,GL_COLOR_BUFFER_BIT,GL_FILTER);


but this gives me a completley black screen instead of completley red. Any suggestions?
I've seen some examples online, where they use a 2D array (instead of a 1D array) to store the pixel data. Since multiple rows of an array are just stored one after the other, I don't think this will make a difference, but just to be sure, can someone confirm this?

genzm19
06-11-2013, 09:18 AM
As the current problem of this thread is no longer related with the subforum and the title topic, I've created a new thread in the right forum.
Anyway, a great thanks to Alfonse Reinheart, who has helped me with the problem and showed me some very interessting possibilities of opengl

http://www.opengl.org/discussion_boards/showthread.php/181907-drawing-individual-pixels-with-opengl?p=1251624#post1251624