demosaic and timestamp video

OpenGL OpenGL: Advanced Coding
1

I am working on a program to read a stream of raw (Bayer CFA) images from disk, demosaic, timestamp, jpeg and save to an output disk.

I am using OpenGL and GLSL to accomplish the first 2 steps: I read the raw image array into a texture, display it in a quad that fills the viewport, and apply a program (vertex and fragment shaders) to do the demosaic and display to the framebuffer. (customized pipeline) That works fine.

I also can write the date and time as text to the bottom of a screen using a bitmap font as a texture from which I pull individual chars to construct the string. (standard pipeline)

Problem comes in when I try to put the timestamp on the image: If the shaders are active (i.e., UseProgram() in place), then the image shows up fine; but, the text is garbled. If I turn off the shaders (i.e., comment out UseProgram()), then the image shows up as a raw mosaic, and the timestamp is fine.

How can I apply the shader only to the image, and then blend in the timestamp?

2

I asked an expert and he said to do it in 2 passes: 1) image render with shader code and 2) timestamp render without. OpenTK code is as below, where ProgramObject is the vertex and fragment shaders for demosaicing the Texture called textureMosaic, and texFont is a class encapsulating the code to render a string from a bitmap font loaded as a Texture. Key point is to save and restore the fixed-function pipeline by bracketing the use of the shader program with PushAttrib/PushClientAttrib and then PopAttrib/PopClientAttrib:


       private void Render()
        {
            GL.Clear(ClearBufferMask.ColorBufferBit);

            GL.MatrixMode(MatrixMode.Modelview);
            GL.LoadIdentity();

            GL.PushAttrib(AttribMask.AllAttribBits);
            GL.PushClientAttrib(ClientAttribMask.ClientAllAttribBits);

            GL.UseProgram(ProgramObject);

            GL.Disable(EnableCap.Blend);
            GL.BindTexture(TextureTarget.Texture2D, textureMosaic);

            GL.Begin(BeginMode.Quads);
            GL.TexCoord2(0.0f, 1.0f); GL.Vertex2(-1.0f, -1.0f);
            GL.TexCoord2(1.0f, 1.0f); GL.Vertex2(1.0f, -1.0f);
            GL.TexCoord2(1.0f, 0.0f); GL.Vertex2(1.0f, 1.0f);
            GL.TexCoord2(0.0f, 0.0f); GL.Vertex2(-1.0f, 1.0f);
            GL.End();

            GL.UseProgram(0);

            GL.PopAttrib();
            GL.PopClientAttrib();

            // this blending makes the background of the letters disappear
            GL.Enable(EnableCap.Blend);   
            GL.BlendFunc(BlendingFactorSrc.SrcAlpha, BlendingFactorDest.OneMinusSrcAlpha);

            string dtText = imageTime.ToString("yyyy-MM-dd hh:mm:ss") + " " + imageTime.Millisecond.ToString().PadLeft(3, '0');
            texFont.WriteStringAt(dtText, 3, 50, 2, 0); // text, heightPercent, xPercent, yPercent, degreesCounterClockwise

            glControl1.SwapBuffers();
        }

Note: He said another way to do it is all in the shaders, but I don’t know GLSL well enough to do that. If you do, please reply with an example.

3

Simple : deactivate shaders when you no longer need them.
http://www.opengl.org/sdk/docs/man/xhtml/glUseProgram.xml

And you will need to explicitely enable texturing, in fixed path mode :
http://www.opengl.org/wiki/GLSL_:_common_mistakes#Enable_Or_Not_To_Enable
ie :


glUseProgram(ProgramObject);
// do the demosaic part

glUseProgram(0);
glEnable(GL_TEXTURING);
// do the timestamp part
4

Ok, so then it’s not necessary to worry about pushing/popping attributes. I deleted those lines and added the enable for texturing; and it works fine, too.

Thank you, OpenGL Lord, for the clarifications.