downsampling texture using FBO

[Nisar Ahmed]

Hello,

I am trying to scaledown a texture using FBO with but i only end cropping it instead of resizing it

What can possibly go wrong? I tried setting viewport & projection matrix but setting it only hides the image. Please guide me through this process and suggest a proper way of scaling down images using OpenGL.

FBOTexture fbo = [self setupFBO:NSMakeSize(texWidth, texHeight) buffer:CVPixelBufferGetBaseAddress(frame)];
CVPixelBufferUnlockBaseAddress(frame, 0);

glClearColor(0.0, 0.0, 0.0, 0.0);
glClear(GL_COLOR_BUFFER_BIT);

glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fbo.fbo);


//glViewport(0, 0, scaledWidth ,scaledHeight);

//glMatrixMode(GL_MODELVIEW);
//glLoadIdentity();
//glMatrixMode(GL_PROJECTION);
//glLoadIdentity();
//glOrtho(0, scaledWidth, 0, scaledHeight, -1.0, 1.0);

glBegin(GL_QUADS);

glTexCoord2i(0, 0); glVertex2i(0, 0);
glTexCoord2i(0, texHeight); glVertex2i(0, scaledHeight);
glTexCoord2i(texWidth, texHeight); glVertex2i(scaledWidth, scaledHeight);
glTexCoord2i(texWidth, 0); glVertex2i(scaledWidth, 0);

glEnd();

//[glContext flushBuffer];
CVPixelBufferLockBaseAddress(resizedFrame, 0);
glReadPixels(0, 0, scaledWidth, scaledHeight, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8, CVPixelBufferGetBaseAddress(resizedFrame));
CVPixelBufferUnlockBaseAddress(resizedFrame, 0);

[thokra]

The dimensions of the FBO attachments and the viewport have to match. Also, reading back with the scaled dimensions isn't what you want if the size of your FBO doesn't match. What you want is to simply read back the whole contents of the color buffer. The downsampling is done by the GL automatically using the specified filtering method. From the top of my head I can come up with something like this:

Let w = scale * width and h = scale * height

a) Setup FBO with a color attachment with dimensions (w, h)
b) specify viewport with (0, 0, w, h)
c) render screen-filling quad and apply texture with tex coords
(0, 0)
(1, 0)
(1, 1)
(0, 1)
d) read back pixels from (0, 0) to (w, h)

Since you're using the fixed-function pipeline, the approach for a full-screen quad described here is sufficient:

http://www.opengl.org/resources/faq/...formations.htm

Check it out and get back.

[Nisar Ahmed]

I can now see the portion of the image, even smaller. I think GL is not downsampling the texture automatically.

Please see the code that generates FBO. the buffer's dimensions are 720*288 and the scaled dimensions would be 360*144. You are suggesting to create a texture with scaled dimensions while the buffer holds 720*288 pixels? and where should I apply filtering so that OpenGL downsamples during readback?

Thank you for help


- (FBOTexture)setupFBONSSize)size bufferPtr)buffer
{

FBOTexture fboTexture;

// texture / color attachment
glGenTextures(1, &fboTexture.texture);
glEnable(GL_TEXTURE_RECTANGLE_EXT);
glBindTexture(GL_TEXTURE_RECTANGLE_EXT, fboTexture.texture);
glTexImage2D(GL_TEXTURE_RECTANGLE_EXT, 0, GL_RGBA8, size.width, size.height, 0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8, buffer);
glBindTexture(GL_TEXTURE_RECTANGLE_EXT, 0);

// FBO and connect attachments
glGenFramebuffersEXT(1, &fboTexture.fbo);
glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fboTexture.fbo);
glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_RECTANGLE_EXT, fboTexture.texture, 0);
glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0);

GLenum status = glCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT);
if(status != GL_FRAMEBUFFER_COMPLETE_EXT)
{
NSLog(@"OpenGL error %04X", status);
glDeleteTextures(1, &fboTexture.texture);
glDeleteFramebuffersEXT(1, &fboTexture.fbo);
fboTexture.texture = 0;
fboTexture.fbo = 0;
}
return fboTexture;
}

[thokra]

You are suggesting to create a texture with scaled dimensions while the buffer holds 720*288 pixels?

No. I'm saying your supposed to create a framebuffer with the target size - the size of the downsampled image. Same for the viewport. Then the attachments of the FBO will have dimensions 360*144.

As far as I can see, you don't set any minification/magnification filters. Use glTexParameter and set the minification filter to GL_LINEAR. BTW, why TEXTURE_RECTANGLE?

[Nisar Ahmed]

I have done exactly what you are saying but the image I am getting after readback is only a portion of the original image, not the full scaled down image!. Also the resulting image is now also repeated / tiled.

There is one more thing that now setting viewport doesn't make any different, the output image is always the same.

I have also added glTexParameter

// texture / color attachment
glGenTextures(1, &fboTexture.texture);

glBindTexture(GL_TEXTURE_2D, fboTexture.texture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, size.width, size.height, 0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8, buffer);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);

glBindTexture(GL_TEXTURE_2D, 0);

[thokra]

If you draw a screen-filling quad and set the texture coordinates as stated above, then it's impossible that the image is repeated. Please post the whole routine, including setup and rendering.

[Nisar Ahmed]

I think its the ATI & NVIDIA issue, it was working ok on NVIDIA and not on ATI

//setupFBO creates FBO the first time with the size of the final resized image
-(void)setupFBONSSize)size bufferPtr)buffer;
//scaleFrame creates new texture based on the new frame and attaches it to the FBO
-(void)scaleFrameCVImageBufferRef)frame toSizeNSSize)size resizedFrameCVImageBufferRef)resizedFrame;
//this is a utility function used to generate texture from source pixels of the final size, if I pass original size the image crops otherwise it gets corrupted. But on NVIDIA it works well with the original size?
-(GLuint)textureFromPixelsCVImageBufferRef)image ofSizeNSSize)size;

- (void)setupFBONSSize)size bufferPtr)buffer //used to setup FBO the first time, buffer is set to NULL is size is the final size
{

CGLContextObj cgl_ctx = [offscreenContext CGLContextObj];

if(fbo.texture)
{
glDeleteTextures(1, &fbo.texture);
}
if(fbo.fbo)
{
glDeleteFramebuffersEXT(1, &fbo.fbo);
fbo.fbo = 0;
}
if(fbo.depthBuffer)
{
glDeleteRenderbuffersEXT(1, &fbo.depthBuffer);
fbo.depthBuffer = 0;
}
// texture / color attachment
glGenTextures(1, &fbo.texture);

glBindTexture(GL_TEXTURE_RECTANGLE_EXT, fbo.texture);
glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_WRAP_T, GL_REPEAT);
glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_RECTANGLE_EXT,
GL_TEXTURE_STORAGE_HINT_APPLE,
GL_STORAGE_SHARED_APPLE);
glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE);
glTexImage2D(GL_TEXTURE_RECTANGLE_EXT, 0, GL_RGBA8, size.width, size.height, 0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, buffer);

glBindTexture(GL_TEXTURE_2D, 0);

/*
// depth buffer
glGenRenderbuffersEXT(1, &fbo.depthBuffer);
glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, fbo.depthBuffer);
glRenderbufferStorageEXT(GL_RENDERBUFFER_EXT, GL_DEPTH_COMPONENT, size.width, size.height);
glBindRenderbufferEXT(GL_RENDERBUFFER_EXT, 0);
*/

// FBO and connect attachments
glGenFramebuffersEXT(1, &fbo.fbo);
glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fbo.fbo);
glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_RECTANGLE_EXT, fbo.texture, 0);
//glFramebufferRenderbufferEXT(GL_FRAMEBUFFER_EXT, GL_DEPTH_ATTACHMENT_EXT, GL_RENDERBUFFER_EXT, fbo.depthBuffer);

glClearColor(0.0, 0.0, 0.0, 0.0);
glClear(GL_COLOR_BUFFER_BIT);

//glPushAttrib(GL_VIEWPORT_BIT);
glViewport(0, 0, size.width ,size.height);

//glPushMatrix();
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
glMatrixMode(GL_PROJECTION);
//glPushMatrix();
glLoadIdentity();
glOrtho(0, size.width, 0, size.height, -1.0, 1.0);

glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0);

GLenum status = glCheckFramebufferStatusEXT(GL_FRAMEBUFFER_EXT);
if(status != GL_FRAMEBUFFER_COMPLETE_EXT)
{
NSLog(@"OpenGL error %04X", status);
glDeleteTextures(1, &fbo.texture);
glDeleteFramebuffersEXT(1, &fbo.fbo);
//glDeleteRenderbuffersEXT(1, &fbo.depthBuffer);
fbo.texture = 0;
fbo.fbo = 0;
fbo.depthBuffer = 0;
}
}
//This function attaches new texture with update frame and render
-(void)scaleFrameCVImageBufferRef)frame toSizeNSSize)size resizedFrameCVImageBufferRef)resizedFrame
{

GLuint texture = [self textureFromPixels:frame ofSize:size];

int texWidth = CVPixelBufferGetWidth(frame);
int texHeight = CVPixelBufferGetHeight(frame);

CGLContextObj cgl_ctx = [offscreenContext CGLContextObj];

glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, fbo.fbo);
glFramebufferTexture2DEXT(GL_FRAMEBUFFER_EXT, GL_COLOR_ATTACHMENT0_EXT, GL_TEXTURE_RECTANGLE_EXT, texture, 0);

glBegin(GL_QUADS);

glTexCoord2f(texWidth, texHeight); glVertex3f(1.0, 1.0, 0); // upper right
glTexCoord2f(0.0, texHeight); glVertex3f(0.0, 1.0, 0); // upper left
glTexCoord2f(0.0, 0.0); glVertex3f(0.0, 0.0, 0); // lower left
glTexCoord2f(texWidth, 0.0); glVertex3f(1.0, 0.0, 0); // lower right

glEnd();

//glPixelStorei(GL_PACK_ALIGNMENT, 4);
//glPixelStorei(GL_PACK_ROW_LENGTH, size.width);
CVPixelBufferLockBaseAddress(resizedFrame, 0);
glReadPixels(0, 0, size.width, size.height, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, CVPixelBufferGetBaseAddress(resizedFrame));
CVPixelBufferUnlockBaseAddress(resizedFrame, 0);

glBindFramebufferEXT(GL_FRAMEBUFFER_EXT, 0);
}
//finally this function creates texture from pixels, the size is the final size not the original
- (GLuint)textureFromPixelsCVImageBufferRef)image ofSizeNSSize)size
{
CGLContextObj cgl_ctx = [offscreenContext CGLContextObj];
int texWidth = size.width; //CVPixelBufferGetWidth(image);
int texHeight = size.height; //CVPixelBufferGetHeight(image);
if(fbo.texture)
{
glDeleteTextures(1, &fbo.texture);
}

//GLuint texture;

glGenTextures(1, &fbo.texture);

glBindTexture(GL_TEXTURE_RECTANGLE_EXT, fbo.texture);
//glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_WRAP_S, GL_REPEAT);
//glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_WRAP_T, GL_REPEAT);
//glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
//glTexParameteri(GL_TEXTURE_RECTANGLE_EXT, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_RECTANGLE_EXT,
GL_TEXTURE_STORAGE_HINT_APPLE,
GL_STORAGE_SHARED_APPLE);
glPixelStorei(GL_UNPACK_CLIENT_STORAGE_APPLE, GL_TRUE);

CVPixelBufferLockBaseAddress(image, 0);
glTexImage2D(GL_TEXTURE_RECTANGLE_EXT, 0, GL_RGBA8, texWidth, texHeight, 0, GL_BGRA, GL_UNSIGNED_INT_8_8_8_8_REV, CVPixelBufferGetBaseAddress(image));
CVPixelBufferUnlockBaseAddress(image, 0);

glBindTexture(GL_TEXTURE_RECTANGLE_EXT, 0);
return fbo.texture;
}