Extracting Y from GL_LUMINANCE

[DrFooMod2]

I have a working fragment shader for translating YUV 420 YCbCr Biplanar iPhone 4 camera frames to RGB. I was able to read enough samples, and the YUV (http://en.wikipedia.org/wiki/YUV) page at Wikipedia to get what I needed:



#ifdef GL_ES
precision mediump float;
#endif

varying vec2 textureCoordinate;

uniform sampler2D videoFrame;
uniform sampler2D videoFrameUV;

const mat3 yuv2rgb = mat3(
1, 0, 1.2802,
1, -0.214821, -0.380589,
1, 2.127982, 0
);

void main() {

vec3 yuv = vec3(
1.1643 * (texture2D(videoFrame, textureCoordinate).r - 0.0625),
texture2D(videoFrameUV, textureCoordinate).r - 0.5,
texture2D(videoFrameUV, textureCoordinate).a - 0.5
);
vec3 rgb = yuv * yuv2rgb;

gl_FragColor = vec4(rgb, 1.0);
}


My question is why the math for the Y, U, and V components? That I couldn't find. Thanks!

FYI, the matrix values are based on Rec. 709.

[Bruce Wheaton]

When you say why the math, do you mean why Y is scaled, and U and V are offset by 0.5? The Y is being scaled there because the code is assuming that video levels are being used, where 16 is picture black (so 'super-black' is supported), and picture white is 235. That math (I didn't check the numbers, but you can) converts to a more normal 0.0 - 1.0. Depending on what your write the result to, it will be clamped.

U and V are color difference signals, so they represent a signed number with a range of approx -0.5 to 0.5. It's most common to offset the signed value to make an unsigned integer in the 0-255 range, representing -127 to 127. The math converts it back to a signed floating point difference value.

Bruce

[DrFooMod2]

That completely answers my question! I recall at some point reading about video frames being 16 to 235, but I didn't put 2 and 2 together. Thanks so much!


Edit:

I went back and found this

http://en.wikipedia.org/wiki/Rec._709#Digital_representation

which clears up the 16 ... 235 point.

[DrFooMod2]

Acha! 16/255 = 0.0625. So, if you have a value of 16, then 1.1643 * 16 - 0.0625 = 0. Conversely, 235/256 = 0.91796 and (0.91796 * 1.1643) - 0.0625 ~= 1.

:D