Part of the Khronos Group
OpenGL.org

The Industry's Foundation for High Performance Graphics

from games to virtual reality, mobile phones to supercomputers

Page 1 of 7 123 ... LastLast
Results 1 to 10 of 66

Thread: Direct GL_YUV, GL_YCbCr or YCoCg texture support ?

  1. #1
    Junior Member Regular Contributor
    Join Date
    Nov 2009
    Location
    French
    Posts
    117

    Direct GL_YUV, GL_YCbCr or YCoCg texture support ?

    Why OPENGL haven't a standardised texture support for standard TV digital video format such as YUV or YCbCr in 4:2:2 or 4:1:1 format ???

    In first view, we have to use a color matrix for this
    => why this haven't to be upgraded since a long time ???

    Alls videos are made with a sort of YUV format, so we have to systematicaly convert it from YUV to RGB before send it to an OPENGL texture => it's a lot of loose time for nothing ...

    Alls recents graphics cards seem have a support for YUV format, so this can't be a technical problem ... and only some add/mul aren't really hard to integrate into a graphic chipset or the API : )

    It's really mysterious for me to understand why this haven't yet already exposed into the OpenGL API (because this format exist since a lot of decades ...)


    @+
    Yannoo
    @+
    Yannoo

  2. #2
    Junior Member Regular Contributor
    Join Date
    Nov 2009
    Location
    French
    Posts
    117

    Re: Direct GL_YUV, GL_YCbCr or YCoCg texture suppo

    It is the same thing with some hypotetics GL_MPEG1, GL_MPEG2 and/or a relatively new GL_MPEG4 supports for video texturing too

    This work already very fine and in real-time with avicodec that decode MPEG1/MPEG2/MPEG4 streams that I can easily put into OPENGL textures ...

    So ... => WHY ???


    @+
    Yannoo
    @+
    Yannoo

  3. #3
    Junior Member Regular Contributor
    Join Date
    Nov 2009
    Location
    French
    Posts
    117

    Re: Direct GL_YUV, GL_YCbCr or YCoCg texture suppo

    I think that the Mesa extension YCBCR_422_MESA is something like I want and that Apple have make a similar (but really slow from what I have can read) extension in the past ...

    I test if this is exposed into my various PCs, EEEPC or Macintoshs that have alls differents graphics cards and drivers ...


    @+
    Yannoo
    @+
    Yannoo

  4. #4
    Senior Member OpenGL Guru Dark Photon's Avatar
    Join Date
    Oct 2004
    Location
    Druidia
    Posts
    3,189

    Re: Direct GL_YUV, GL_YCbCr or YCoCg texture support ?

    Quote Originally Posted by Yann LE PETITCORPS
    Why OPENGL haven't a standardised texture support for standard TV digital video format such as YUV or YCbCr in 4:2:2 or 4:1:1 format ???

    ...

    It's really mysterious for me to understand why this haven't yet already exposed into the OpenGL API (because this format exist since a lot of decades ...)
    OpenGL is a 2D/3D graphics rendering API, not a video processing API. Is that to say you can't do video processing using OpenGL? No. But it's to say that there aren't custom APIs geared around it.

    I suggest you check out APIs such as the X.Org Xv library, which already has built-in support for playback with these video formats, with GPU acceleration when available, and APIs such as NVidia's VDPAU, which provides GPU-assisted decode and playback of even very compute-intensive formats like h.264.

  5. #5
    Junior Member Newbie
    Join Date
    Sep 2003
    Posts
    9

    Re: Direct GL_YUV, GL_YCbCr or YCoCg texture suppo

    A number of vendors (i.e. Apple, ...) have had support for rendering YCbCr 4:2:2 formats through the OpenGL API. For example on Mac OS X (going all the way back to 10.2.x) you have support for GL_APPLE_ycbcr_422 which allows one to read, store and optionally process texture data in (signed, unsigned) YCbCr 4:2:2. Considering that consumer HW has had direct (texture) support for 4:2:2 for quite some time the performance cost of this particular extension is effectively nonexistent.

    Snow Leopard gives you more control over the server side conversion from YCbCr into RGB by way of GL_APPLE_rgb_422. The latter extension allows one to generate their own (client side) conversion from YCbCr into RGB via the programmable pipeline or through another method.

    If you need support for other YCbCr formats (i.e. 4:2:0, 4:4:4, ...) you are much better off using the existing (L, A, RGB) texture path and the programmable pipeline to accomplish this.

  6. #6
    Advanced Member Frequent Contributor yooyo's Avatar
    Join Date
    Apr 2003
    Location
    Belgrade, Serbia
    Posts
    872

    Re: Direct GL_YUV, GL_YCbCr or YCoCg texture suppo

    YUV422, 411, 420 and all sort of it is easy to implement via shaders, and render targets. For example to perform YUV422 to RGB you need:
    1. RGBA texture contains YUV422 samples
    2. FBO with RGB target texture bound
    3. draw screen aligned textured quad with appropriate fragment shader
    In shader you need to resolve sampling (422 in this case), collect YUV samples, convert to RGB (using conversion matrix provided by app) and write RGB pixels.

    Then just use RGB version of texture.

    In above example I choose RGBA texture to store YUV422 samples. YUV422 format looks like:
    Code :
    Y0 U Y1 V  Y0 U Y1 V  Y0 U Y1 V... and it perfectly fits in
    R  G B  A  R  G B  A  R  G B  A
    So.. for a given YUV422 @ width x height, create width/2 x height RGBA texture and just fill raw YUV422 data. Also create target RGB texture width x height and bind it to FBO.

    In shader (depending on even/odd gl_FragCoord.x coordinate) choose Y0 U V or Y1 U V and perform YUV to RGB conversion.

    Other format and samplings is easy job to do.

  7. #7
    Super Moderator Frequent Contributor Groovounet's Avatar
    Join Date
    Jul 2004
    Posts
    934

    Re: Direct GL_YUV, GL_YCbCr or YCoCg texture suppo

    I first thought: Great idea! And then I realize that you can store the YUV texture into several textures and decode them within a shader.

    Maybe a dedicated format could be more efficient on the GPU side but I get the limitation will be on the image streaming.

  8. #8
    Member Regular Contributor
    Join Date
    Nov 2003
    Location
    Czech Republic
    Posts
    317

    Re: Direct GL_YUV, GL_YCbCr or YCoCg texture suppo

    I am pretty against the idea to have YUV support in OpenGL.
    I can understand that in 2.x era. But not today with all the shader functionality.

    The problem with YCrCb 422 & co. is that there are so many combinations or memory format. (CrYCbY, YCrYCb,....)

    There are several color matrices to convert to RGB (CCIR601, CCIR709, FullRange).

    Then you can do have several ways to upscale from 422 to 444. Using simple upscale or linear filtered upscale or more sophisticated upscaling.

    Then you have 8, 10, 12 bits per channel.

    Hardwiring all this into driver is pretty agains the idea to have few fixed func. features and more programmable features.

    I'd more understand to make some utility library to handle all this.

  9. #9
    Junior Member Regular Contributor
    Join Date
    Jan 2004
    Location
    Czech Republic, EU
    Posts
    190

    Re: Direct GL_YUV, GL_YCbCr or YCoCg texture suppo

    If hardware supports it directly, there is no reason for it not to be included in the API. It's in Apple's implementation and Mesa, so it probably should be standardized. It's worth noting that Xv uses 3D hardware these days anyway.
    (usually just hobbyist) OpenGL driver developer

  10. #10
    Super Moderator Frequent Contributor Groovounet's Avatar
    Join Date
    Jul 2004
    Posts
    934

    Re: Direct GL_YUV, GL_YCbCr or YCoCg texture suppo

    The video decoder is a separed component from the GPU for some company. I don't know about ATI or nVidia but even if it's on the same chip it doesn't means that the video decoded is really integrated with the rest of the chip. "It lives by it's own".

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •