How do I use multiple textures properly?

[Bjorn]

My application displays two rectangles. Both rectangles are just a regular GL_QUAD rectangle with a texture mapped image. One of the windows texture map an RGBA image, the other window texture maps a three-texture YUV420 image. The YUV image uses a fragment shader to do color conversion.

If I display only one of the rectangles, the image looks fine. The RGBA image looks fine if that rectangle is the only image displayed and the YUV image looks fine if that's the only rectangle displayed.

Trouble starts when I try to display both images at the same time. The RGBA image becomes corrupted. It suddenly contains only one of the planes and looks awfully green. The pixels are correct, but the color is way off.

The problem seems to be related to the use and reuse of textures. The two "windows" have separate rendering functions and do not share texture id's. Each rendering function calls glGenTextures() and glDeleteTextures().

Some of the properties set for the textures seems to be retained. Here's a snippet describing the structure of my code:

// First draw the YUV image
glGenTextures(3, ptr); // 3 textures for YUV rendering
...
glTexImage2D(..., GL_LUMINANCE, bytes_per_pixel, ...);
...
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texid[0]);
glTexSubImage2D(...,GL_LUMINANCE, bytes_per_pixel, y);
...
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, texid[1]);
glTexSubImage2D(...,GL_LUMINANCE, bytes_per_pixel, u);
...
glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D, texid[2]);
glTexSubImage2D(...,GL_LUMINANCE, bytes_per_pixel, v);
...
glBegin(GL_QUADS);
draw window using glMultiTexCoord + glVertex3f
glEnd()
glFlush();
glDeleteTextures();


// Then draw the RGBA image, with 4 bytes per pixel
glGenTextures(1, &texid);
glBindTexture(GL_TEXTURE_2D, texid);
glActiveTexture(GL_TEXTURE0);
...

glTexImage2D(..., GL_RGBA, GL_UNSIGNED_BYTE, rgbdata);
...
glBegin(GL_QUADS);
draw window using glMultiTexCoord + glVertex3f
glEnd()
glFlush();
glDeleteTextures();

What am I doing wrong here? Why does the re-use of GL_TEXTURE0 inherit properties from the first use of GL_TEXTURE0?

[Dark Photon]

Trouble starts when I try to display both images at the same time. The RGBA image becomes corrupted. It suddenly contains only one of the planes and looks awfully green. The pixels are correct, but the color is way off.
Post an image. What makes you say that they are corrupt?

Also, are you talking about drawing two quads within the same OS window, or two quads, each in different OS windows. You keep using the term "window" in a confusing way.

Also, what do you "want" to have happen when these quads overlap. Do you want them to blend or not? If yes, have you enabled blending, are you setting alpha values for each quad, and are you drawing them back-to-front? Show us your rendering code, not just the texture creation.

[Bjorn]

Sorry for the confusing terminology.

The quads do not overlap. They're "windows" displayed within a GLX window. A live video stream is displayed inside each "window"/quad, and the quads are placed side by side.

I'll have to get back to you later regarding the image. Didn't even know that I could post an image..

[Bjorn]

Here's a couple of images illustrating the problem. The leftmost image is RGBA, the rightmost is YUV. It is not the same image, the input is from two separate webcams pointing in the same direction.

http://picasaweb.google.no/bjorn.augestad/OpenGL#

[strattonbrazil]

Well, if you're rendering two quads (or two "windows" or whatever you want to call them), why are you using multitexturing? Why don't you just render one, switch textures, then render the other? If they never overlap you probably don't need multitexturing? Were you expecting a speedup using just one glBegin call or something?

[Bjorn]

I'm using multitexturing because my YUV->RGB color space converter works with three textures. So the rightmost window/quad is rendered with 3 textures and a shader.

The leftmost window/quad is rendered with one texture.

The windows/quads are part of a framework I'm creating. This framework can have any number of output windows/quads. The framework initially had support for YUV source data only, but worked perfectly. When I added support for RGBA source data, I got into trouble.

Hope this makes sense to you guys...

[Dark Photon]

If I display only one of the rectangles, the image looks fine. The RGBA image looks fine if that rectangle is the only image displayed and the YUV image looks fine if that's the only rectangle displayed.

Trouble starts when I try to display both images at the same time. The RGBA image becomes corrupted.
Ok. Given what you've said, sounds like you've got some OpenGL state not being reset/set properly when you render both images at the same time, OR your decoder is fouling up when it's processing both images at the same time. With your rendering code we could say more, but alas...

Check the state of your OpenGL state before drawing the RGBA image when drawing one image. Compare to when you're drawing both. Of course you only need to check when you modify in your code.

Also, try reversing the draw order of the two images and see if your artifact "moves" from one image to the other. That may give you some clues.

Also, try glPushAttrib/glPopAttrib before/after images to help isolate if/which state you might not be setting properly.

[Bjorn]

Here's the code I use to draw the quads. The draw_stuff() function iterates over a list of "windows" and calls draw_video_window(), which may call draw_rgba_window().

I've noticed that the indentation is removed when posting. If anyone knows how to post code with retained indentation, please let me know.

Thanks again to all for spending time on this problem.

Bjørn


struct glstate {
Window window;
Display *display;
GLXContext ctx;
XVisualInfo *xvi;
GLXFBConfig *fbc;

GLuint yuv420_shader;
};

static void create_textures(GLuint *texid, const struct videobuffer *b)
{
size_t w, h, luma_w, luma_h;
int bytes_per_pixel = GL_UNSIGNED_BYTE;
w = videobuffer_width(b);
h = videobuffer_height(b);
luma_w = videobuffer_width(b) / 2;
luma_h = videobuffer_height(b) / videobuffer_format_ratio(b);

glGenTextures(3, texid);
check_gl_error();

if(videobuffer_format(b) == TMK_VIDEO_FORMAT_YUV422P16)
bytes_per_pixel = GL_UNSIGNED_SHORT;

glBindTexture(GL_TEXTURE_2D, texid[0]);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, w, h, 0, GL_LUMINANCE, bytes_per_pixel, NULL);
check_gl_error();

glBindTexture(GL_TEXTURE_2D, texid[1]);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, luma_w, luma_h, 0, GL_LUMINANCE, bytes_per_pixel, NULL);
check_gl_error();

glBindTexture(GL_TEXTURE_2D, texid[2]);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, luma_w, luma_h, 0, GL_LUMINANCE, bytes_per_pixel, NULL);
check_gl_error();
}

static void draw_rgba_window(struct widmap *wnd, struct videobuffer *b)
{
coord_t x, y, w, h;
GLuint texid;
void * ydata;
float right = 1.0, top = 1.0;
float left = 0.0, bottom = 0.0;
int type = GL_UNSIGNED_BYTE;

w = videobuffer_width(b);
h = videobuffer_height(b);
x = composer_window_xpos(wnd->wnd);
y = composer_window_ypos(wnd->wnd);
ydata = videobuffer_y(b);

glPixelStorei(GL_UNPACK_ALIGNMENT, 1);

glGenTextures(1, &texid);
glBindTexture(GL_TEXTURE_2D, texid);

glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, w, h, 0, GL_RGBA, type, ydata);

glBindTexture(GL_TEXTURE_2D, texid);

glBegin(GL_QUADS);
glTexCoord2f(left, top); glVertex3f(x, y , 0);
glTexCoord2f(right, top); glVertex3f(x + w, y, 0);
glTexCoord2f(right, bottom); glVertex3f(x + w, y + h, 0);
glTexCoord2f(left, bottom); glVertex3f(x, y + h, 0);
glEnd();

glFlush();
glDeleteTextures(1, &texid);
check_gl_error();
}

static void draw_video_window(struct widmap *wnd, struct glstate *pgl)
{
// Pointers into the buffer data
void *ydata, *udata, *vdata;
struct videobuffer *b;
int bytes_per_pixel = GL_UNSIGNED_BYTE;

coord_t x, y, w, h;
GLint tloc;
GLuint texture_ids[3];
float right = 1.0, top = 1.0;
float left = 0.0, bottom = 0.0;

if(wnd->wnd == NULL)
return;

composer_window_execute_tasklets(wnd->wnd);
if(composer_window_ishidden(wnd->wnd))
return;

// Do we have a buffer for this window?
if( (b = wnd->last_buffer_published) == NULL)
return;

// Is the buffer available for use?
if(videobuffer_get(b) != 0)
return;

x = composer_window_xpos(wnd->wnd);
y = composer_window_ypos(wnd->wnd);
w = composer_window_width(wnd->wnd);
h = composer_window_height(wnd->wnd);

/* If we have a RGBA buffer, we don't need to run the shader.
* We just need to texture map one image. */
if(videobuffer_format(b) == TMK_VIDEO_FORMAT_RGBA) {
draw_rgba_window(wnd, b);

videobuffer_addref(b);
videobuffer_release(b);
return;
}

// Create the textures
create_textures(texture_ids,b);

glUseProgram(pgl->yuv420_shader);
check_gl_error();

// Figure out where the video data is within the buffer
ydata = videobuffer_y(b);
udata = videobuffer_u(b);
vdata = videobuffer_v(b);
if(videobuffer_format(b) == TMK_VIDEO_FORMAT_YUV422P16)
bytes_per_pixel = GL_UNSIGNED_SHORT;

// Connect shader with our pointers
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture_ids[0]);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0,
videobuffer_width(b),
videobuffer_height(b),
GL_LUMINANCE, bytes_per_pixel, ydata);
tloc = glGetUniformLocation(pgl->yuv420_shader, "ytex");
if(tloc == -1)
warning("Could not locate ytex\n");
glUniform1i(tloc, 0);
check_gl_error();

glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, texture_ids[1]);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0,
videobuffer_width(b) / 2,
videobuffer_height(b) / videobuffer_format_ratio(b),
GL_LUMINANCE, bytes_per_pixel, udata);
tloc = glGetUniformLocation(pgl->yuv420_shader, "utex");
if(tloc == -1)
warning("Could not locate utex\n");
glUniform1i(tloc, 1);
check_gl_error();

glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D, texture_ids[2]);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0,
videobuffer_width(b) / 2,
videobuffer_height(b) / videobuffer_format_ratio(b),
GL_LUMINANCE, bytes_per_pixel, vdata);
tloc = glGetUniformLocation(pgl->yuv420_shader, "vtex");
if(tloc == -1)
warning("Could not locate vtex\n");
glUniform1i(tloc, 2);
check_gl_error();

if(composer_window_ismirrored(wnd->wnd)) {
/* Flip the texture mapping */
right = 0.0;
left = 1.0;
}

// Now draw a rectangle and map our 3 textures
// to the 4 corners of the rectangle.
glBegin(GL_QUADS);
glMultiTexCoord2f(GL_TEXTURE0, left, top);
glMultiTexCoord2f(GL_TEXTURE1, left, top);
glMultiTexCoord2f(GL_TEXTURE2, left, top);
glVertex3f(x, y , 0);

glMultiTexCoord2f(GL_TEXTURE0, right, top);
glMultiTexCoord2f(GL_TEXTURE1, right, top);
glMultiTexCoord2f(GL_TEXTURE2, right, top);
glVertex3f(x + w, y, 0);

glMultiTexCoord2f(GL_TEXTURE0, right, bottom);
glMultiTexCoord2f(GL_TEXTURE1, right, bottom);
glMultiTexCoord2f(GL_TEXTURE2, right, bottom);
glVertex3f(x + w, y + h, 0);

glMultiTexCoord2f(GL_TEXTURE0, left, bottom);
glMultiTexCoord2f(GL_TEXTURE1, left, bottom);
glMultiTexCoord2f(GL_TEXTURE2, left, bottom);
glVertex3f(x, y + h, 0);
glEnd();

check_gl_error();

glFlush();
// Free the textures after use
glDeleteTextures(3, texture_ids);
check_gl_error();

// Release the buffer as we don't need it anymore,
// but add a reference to it in case we don't get
// a new version when it is time to draw next frame.
videobuffer_addref(b);
videobuffer_release(b);
}

static void draw_stuff(composer frame,
struct widmap * windows,
size_t nwindows,
struct glstate *pgl)
{
size_t i;

glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(
0, composer_width(frame),
0, composer_height(frame),
10, -10);

// Clear matrix stack
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();

glClearColor(0.5, 0.5, 0.5, 1.0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glEnable(GL_TEXTURE_2D);

for(i = 0; i < nwindows; i++) {
/* Later we can add different window types. */
draw_video_window(&amp;windows[i], pgl);
}

// Display rendering
glXSwapBuffers(pgl->display, pgl->window);
check_gl_error();
}

static void create_x_window(composer frame, struct glstate *pgl)
{
// Specify the config we want
XSetWindowAttributes attr;
GLint mask = CWBorderPixel | CWBitGravity | CWEventMask| CWColormap;
int nelements;
int attrib_list[] = {
GLX_RGBA,
GLX_DOUBLEBUFFER,
GLX_RED_SIZE, 8,
GLX_GREEN_SIZE, 8,
GLX_BLUE_SIZE, 8,
//GLX_RENDER_TYPE, GLX_RGBA_BIT,
//GLX_X_RENDERABLE, True,
GLX_DRAWABLE_TYPE, GLX_WINDOW_BIT,
GLX_CONFIG_CAVEAT, GLX_NONE,
0};

// Start means:
// a) create the glx/X window to render to
// b) Start logging buffer id's for windows
// c) Start responding to ticks from our timer so we know
// when to render.
if( (pgl->display = XOpenDisplay(NULL)) == NULL)
die("Could not open display %s\n", getenv("DISPLAY"));

pgl->fbc = glXChooseFBConfig(pgl->display, 0, attrib_list, &amp;nelements);
if(pgl->fbc == NULL)
die("Could not get FBConfig\n");

pgl->xvi = glXChooseVisual(pgl->display, DefaultScreen(pgl->display), attrib_list);
if(pgl->xvi == NULL)
die("Could not get visual from fb config\n");

// Setup window attributes
attr.event_mask
= ExposureMask
| VisibilityChangeMask
| KeyPressMask
| PointerMotionMask
| StructureNotifyMask;
attr.border_pixel = 0;
attr.bit_gravity = StaticGravity;
attr.colormap = XCreateColormap(pgl->display,
RootWindow(pgl->display, pgl->xvi->screen),
pgl->xvi->visual,
AllocNone);

// Create a window
pgl->window = XCreateWindow(pgl->display,
DefaultRootWindow(pgl->display), // parent
0, 0, // x,y
composer_width(frame),
composer_height(frame),
0, // border width
pgl->xvi->depth, // depth
InputOutput, // class
pgl->xvi->visual, // Visual
mask, // valuemask
&amp;attr); // attributes

XMapWindow(pgl->display, pgl->window);

pgl->ctx = glXCreateContext(pgl->display, pgl->xvi, 0, True);
glXMakeCurrent(pgl->display, pgl->window, pgl->ctx);

// Initialize GL
glViewport(0, 0, composer_width(frame), composer_height(frame));
glScissor(0, 0, composer_width(frame), composer_height(frame));
glClearColor(0, 0, 0, 1);
glClear(GL_COLOR_BUFFER_BIT);

// Clear matrix stacks
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();

// Compile the 420 color conversion shader
pgl->yuv420_shader = create_420_conversion_shader();
}

[Dark Photon]

Ok, try the ideas I suggested first and let us know.

[Bjorn]

Ok, try the ideas I suggested first and let us know.


I've tried to push and pop each and every attibute possible, to no avail. I've inverted the drawing order of the quads, didn't help. Tried to push and pop the Texture Matrix, no success.

My main PC uses ATI Radeon 4890 with the latest drivers. The same error occurs on a different machine with NVIDIA HW. I guess this means that the error isn't in the fglrx-driver.

More ideas are welcome ... :-)

[Bjorn]

FYI: Writing to let you know that the problem is solved. The code posted above has 2 bugs. a) The RGB texture code does not disable the YUV shader, so I added a glUseProgram(0), b) The RGB texture code does not call glActiveTexture().