Extracting Y from GL_LUMINANCE

I have a working fragment shader for translating YUV 420 YCbCr Biplanar iPhone 4 camera frames to RGB. I was able to read enough samples, and the YUV page at Wikipedia to get what I needed:


#ifdef GL_ES
precision mediump float;
#endif

varying vec2 textureCoordinate;

uniform sampler2D videoFrame; 
uniform sampler2D videoFrameUV;

const mat3 yuv2rgb = mat3(
                            1, 0, 1.2802,
                            1, -0.214821, -0.380589,
                            1, 2.127982, 0
                            );

void main() { 
    
    vec3 yuv = vec3(
                    1.1643 * (texture2D(videoFrame, textureCoordinate).r - 0.0625),
                    texture2D(videoFrameUV, textureCoordinate).r - 0.5,
                    texture2D(videoFrameUV, textureCoordinate).a - 0.5
                    );
    vec3 rgb = yuv * yuv2rgb;
    
    gl_FragColor = vec4(rgb, 1.0);
}

My question is why the math for the Y, U, and V components? That I couldn’t find. Thanks!

FYI, the matrix values are based on Rec. 709.

When you say why the math, do you mean why Y is scaled, and U and V are offset by 0.5? The Y is being scaled there because the code is assuming that video levels are being used, where 16 is picture black (so ‘super-black’ is supported), and picture white is 235. That math (I didn’t check the numbers, but you can) converts to a more normal 0.0 - 1.0. Depending on what your write the result to, it will be clamped.

U and V are color difference signals, so they represent a signed number with a range of approx -0.5 to 0.5. It’s most common to offset the signed value to make an unsigned integer in the 0-255 range, representing -127 to 127. The math converts it back to a signed floating point difference value.

Bruce

That completely answers my question! I recall at some point reading about video frames being 16 to 235, but I didn’t put 2 and 2 together. Thanks so much!

Edit:

I went back and found this

http://en.wikipedia.org/wiki/Rec._709#Digital_representation

which clears up the 16 … 235 point.

Acha! 16/255 = 0.0625. So, if you have a value of 16, then 1.1643 * 16 - 0.0625 = 0. Conversely, 235/256 = 0.91796 and (0.91796 * 1.1643) - 0.0625 ~= 1.

:smiley:

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.