PDA

View Full Version : Lock the Default Framebuffer



Alfonse Reinheart
08-26-2012, 05:19 AM
Currently, the default framebuffer can't be used in many of the ways that textures or renderbuffers can be. You can render to them, copy to them with glBlitFramebuffer, but you can't do things like bind them as images or textures and so forth.

One of the reasons for this is that the default framebuffer can be resized by the windowing system. Images need to have a specific, fixed size, so having them be resized can be an issue.

To deal with this issue, I propose the ability to lock images within the default framebuffer. This will prevent the framebuffer from being resized. If a window is resized, then something implementation-dependent will happen. It may be rescaled, it may fill empty areas with garbage, whatever.

It would be simple:



glLockDefaultFramebuffer();


While the default framebuffer is locked, you can get a texture object for the various images. As such:



glTextureDefaultFramebuffer(GL_BACK);


This will return a texture view of the 2D storage for the back buffer of the default framebuffer. So it's immutable. Every call to this will return a new texture object (from an unused texture name, as if from `glGenTextures`). So repeated calls will create new views to the same storage. Obviously, if the framebuffer is not locked, this will fail with an error.

The sticky point is this:



glUnlockDefaultFramebuffer();


Obviously, we would need that function. But what exactly does it mean? What happens to those texture views that were created? The simplest thing would be for any use of them (besides deletion) after unlocking the default framebuffer to become undefined behavior.

An alternative implementation of this concept that avoids the pesky unlock issue is to add a WGL/GLX_CONTEXT_FIXED_BIT_ARB to the attribute flags. If this flag is set, then the context has a size fixed at the time of its creation. In such a context, `glTextureDefaultFramebuffer` will return a valid texture object as defined above; if the context wasn't created with this flag, it returns 0.

kRogue
08-26-2012, 12:22 PM
In my opinion, something like this likely belongs in the land of EGL/GLX/WGL. For example, in EGL land the logic would be like this:

Create EGLImage from framebuffer
Create texture (or renderbuffer or whatever) from EGLImage


In locking the framebuffer, then one would also need to introduce language for rendering to the framebuffer and not using it as a source, unless you are to only allow for sourcing from the front buffer, which in turn assumes the system uses double buffering (some use single buffering, some use triple buffering.. the iOS platform is "interesting", one makes a renderbuffer and uses an iOS call to say that is the contents of the program to display). Along these lines, additional language is required to make sure that if you hook a texture to the front buffer then the texture is read only (because the windowing system is presenting that buffer). Moreover, in EGL land there is a config bit which when enabled basically says that the contents of the framebuffer are undefined after eglSwapBuffers (this is a performance benefit for all tiled based renderers).

Out of curiosity, why do you want this? Along those lines, why not just "fake" your own framebuffer via FBO's and at the end of the frame blit your renderbuffer/texture to the framebuffer?

thokra
08-27-2012, 01:52 AM
I think Alfonse's complaint in regards to the inflexibility of the default FB is justified. However, personally I've been using the default FB like kRogue suggests for quite some time, i.e. only as the target of the final blit. I have the feeling that introducing new APIs and making changes to the window system bindings to achieve what we can already do in a different way should be warranted by some really convincing use cases.

In regards to what kRogue calls "faking" your own framebuffer: In GL ES you can already drop the whole framebuffer completely and be mandated to use FBOs entirely. Of course, this will only give you the opportunity to render off-screen but it's essentially the same except for this annoying missing surface to blit to in the end. Now, providing that one surface will still allow on-screen rendering and window system handled resizing, all the other stuff would be the developers responsibility like swapping and resizing FBO accordingly - unless they want rendering to happen at lower resolution and having an interpolated blit to some higher/lower resolved surface. No depth, no stencil - just a plain RGB(A) surface. This adds complexity to the implementation of the application but would leave at least the GL unchanged, though it would probably necessitate some changes to the window systems bindings.

In conclusion: Give us something that's almost like OES_surfaceless_context (http://www.khronos.org/registry/gles/extensions/OES/OES_surfaceless_context.txt) and EGL_KHR_surfaceless_context (http://www.khronos.org/registry/egl/extensions/KHR/EGL_KHR_surfaceless_context.txt) - only with a single RGB(A) surface we can blit to after rendering has been completed.

mhagain
09-04-2012, 04:40 PM
If a window is resized, then something implementation-dependent will happen. It may be rescaled, it may fill empty areas with garbage, whatever.

This is one major objection I have here. Defining implementation-dependent behaviour is dangerous - some implementations are going to crash, some are going to give garbage, some give black, some rescale, and some may even give what appears to be correct behaviour. That opens huge potential for programmer error. If resizing the framebuffer is to be disallowed then it should have well-defined error behaviour that all implementations must abide by in order to be conformant, and that should be testable-for in code so that the programmer can detect it happening and respond accordingly.

Alfonse Reinheart
09-04-2012, 06:01 PM
This is one major objection I have here. Defining implementation-dependent behaviour is dangerous - some implementations are going to crash, some are going to give garbage, some give black, some rescale, and some may even give what appears to be correct behaviour. That opens huge potential for programmer error. If resizing the framebuffer is to be disallowed then it should have well-defined error behaviour that all implementations must abide by in order to be conformant, and that should be testable-for in code so that the programmer can detect it happening and respond accordingly.

Implementation-dependent behavior doesn't mean crashing is acceptable. Indeed, even undefined behavior doesn't mean that crashing is on the table; the only time crashing is an acceptable option is if the spec explicitly states that program termination is a legitimate outcome.

If you lock the framebuffer, it is on you, and your window-system API of choice, to ensure that the window is not resized. OpenGL should not have any means to tell you it's been resized, since your window system API already lets you know.

l_belev
09-05-2012, 06:18 AM
my opinion is that the default FB should go. it's useless remnant from the pre-fbo times.

a while ago i gave some example about how this can be done and recently i learned that apple have done exactly that in ios. there is no default fb, but instead one creates a special renderbuffer that is displayable. this is done by substituting glRenderbufferStorage with a special non-gl function. from the POV of opengl this special renderbuffer is indistinguishable from a regular one. It is special only for the outside (non-gl) apis that take care of the display.

some of the benefits of getting rid of the default fb are:
- flexibility (the default fb is immutable)
- scrap the redundant non-gl apis for configuring the default fb (pixel formats, visuals, etc).
- makes the glue apis (wgl, glx, etc.) thinner hence opengl less dependent on them.
- does not need any additions to the opengl itself and hence does not complicates it.

just take example from apple (i hope they dont have it patented)

mhagain
09-05-2012, 11:49 AM
Implementation-dependent behavior doesn't mean crashing is acceptable. Indeed, even undefined behavior doesn't mean that crashing is on the table; the only time crashing is an acceptable option is if the spec explicitly states that program termination is a legitimate outcome.

Ignore the example of crashing and focus on the actual point being made, please.

Alfonse Reinheart
09-05-2012, 12:35 PM
Ignore the example of crashing and focus on the actual point being made, please.

If it's not going to crash, then there is nothing "dangerous" about it, and therefore, there is no specific need to have a defined behavior. The reason why the spec allows certain behavior to be implementation defined is to allow for the freedom of optimizing things as is best for that hardware. By defining it in a particular way, you remove the chance for those optimizations.

And I see no need to do that once crashing is off the table. So what if some implementations display garbage or stretch the screen or whatever. The users should not be relying on any of those behaviors, because the user entered into a contract with OpenGL that says that the window will not be resized. That's what locking ultimately means.

Also, please make sure that if you're going to give examples, give reasonable ones. Because your idea will be judged based on the importance of those examples.

kRogue
09-06-2012, 02:36 PM
As a side note, I concur with Illian that iOS's way of doing bits: make a render buffer and use FBO jazz to render to that is better than that currently in EGL/GLX/WGL. Though one will still need a system-dependent call to say "present this render buffer as my window/fullscreen contents" There is a reason why one may want to have the system give you the render target: make the render targets format match the current display settings so presenting the buffer can be done by a blit performed by a dedicated blitter (for example in STE-U8500, the B2R2 unit). Right now though, EGL is a bit of mess (EGLConfig has within what kind of GL context one is to make and that config is needed to get the EGLSurface). But oh well, EGL could be worse it could be WGL.. which reminds me of one of the most hilarious bits in an extension that I have read in a long time (from http://www.opengl.org/registry/specs/NV/float_buffer.txt GL_NV_float_buffer )




Additions to the WGL Specification

First, close your eyes and pretend that a WGL specification actually existed. Maybe if we all concentrate hard enough, one will magically appear.

Eosie
09-06-2012, 03:28 PM
I guess you could make a texture object from a framebuffer even without locking. It wouldn't be so easy, but it would certainly be possible. Or you could just say that the texture object created with glTextureDefaultFramebuffer isn't changed when the default framebuffer is resized. In other words, the texture object would keep referencing the same underlying texture despite the fact it's not part of the default framebuffer anymore. Such a solution would be a lot cleaner than using the lock.

Also unlike FBOs, the default framebuffer can be rendered upside-down. You'd have to add a query returning how it's rendered for the user to know.

Alfonse Reinheart
09-06-2012, 04:45 PM
As a side note, I concur with Illian that iOS's way of doing bits: make a render buffer and use FBO jazz to render to that is better than that currently in EGL/GLX/WGL.

I'm not saying that it would be bad to have that. But it is simply not going to happen, so there's no point in asking for it. In order to make it work in the context of WGL, you have to remove the default framebuffer more or less entirely. And while OpenGL itself is ready to do that (3.0 added language for what happens when there is no default framebuffer), WGL is not.


I guess you could make a texture object from a framebuffer even without locking. It wouldn't be so easy, but it would certainly be possible. Or you could just say that the texture object created with glTextureDefaultFramebuffer isn't changed when the default framebuffer is resized. In other words, the texture object would keep referencing the same underlying texture despite the fact it's not part of the default framebuffer anymore. Such a solution would be a lot cleaner than using the lock.

Cleaner to use, perhaps; not necessarily cleaner to implement. That would mean that, when the window is resized, the implementation needs to suddenly cleave this memory from the default framebuffer. That's probably not an easy thing to just do.

Eosie
09-08-2012, 06:11 PM
Cleaner to use, perhaps; not necessarily cleaner to implement. That would mean that, when the window is resized, the implementation needs to suddenly cleave this memory from the default framebuffer. That's probably not an easy thing to just do.
I've got no idea what you are talking about. :) Buffers, framebuffers, textures... all are just buffers and can be anywhere in memory. Resizing a framebuffer is just a buffer re-allocation, nothing more, nothing less. What I described is very easy to implement. The texture object would just reference the underlying buffer... and that's it. The framebuffer could be released completely, the OpenGL context could even be destroyed... but the kernel wouldn't release the buffer, because I own the reference to it and I could use it anywhere I want, even in the contexts I have not created yet. Or I can just map it (using the kernel interface) and read its memory...

Alfonse Reinheart
09-08-2012, 10:51 PM
Resizing a framebuffer is just a buffer re-allocation, nothing more, nothing less.

Allow me to quote myself:


This will return a texture view of the 2D storage for the back buffer of the default framebuffer. So it's immutable.

There's a reason why texture_storage exists. There's a reason why texture_view requires immutable textures.

If the image you get back can undergo "buffer re-allocation" at any time, then not only can you not rely on its basic properties (size, etc), you also can't use it with texture_view and any other nifty techniques that come along that require immutable texture storage.

"buffer re-allocation" is not a good thing.

Eosie
09-09-2012, 02:33 PM
I don't understand why you are telling me all this. Maybe I was not clear enough, so I'll try again.

Let's say the back buffer is Buffer 1. The moment the texture view is created, both the back buffer and the texture view point to Buffer 1 (it has 2 references at that point). When the window is resized, the back buffer is reallocated, which means Buffer 1 is unreferenced and another buffer is created, let's call it Buffer 2, which is immediately used as a backing storage for the back buffer. Then, the back buffer points to Buffer 2, but the texture view still points to Buffer 1 (it has only one reference now). The texture view pretty much becomes an ordinary immutable texture and has nothing to do with the default framebuffer anymore.

Alfonse Reinheart
09-09-2012, 03:01 PM
The texture view pretty much becomes an ordinary immutable texture and has nothing to do with the default framebuffer anymore.

What good is that compared to simply leaving it undefined? In both cases, in order for the user to actually make useful use of that texture, they have to detect that this has happened, unlock the framebuffer, and lock it again. Since they're going to do that anyway to get a useful result, why bother with stating that this must be how it's implemented?

Or more to the point, what if the implementation allocated a larger backbuffer than you asked for and you resize it? Or better yet, what if the size got smaller on a resize? In either case, the resize doesn't require buffer reallocation; it would simply use a portion of the buffer's true size. But if you force buffer reallocation by requiring it in the spec, then you've forced a heavy-weight operation (buffer reallocation is not cheap) where one was simply not needed.

The user still has to do an unlock/lock cycle as before, because in both cases they can't rely on it (in mine, it's implementation-defined. In yours, it's a guarantee of reallocation). But only in your case does the driver have to do this extra work for no reason.

l_belev
09-10-2012, 02:55 AM
@kRogue: I assume by Illian you mean me?
One more point, we could also have "viewable" textures by e.g. substituting glTexStorage2D with another special non-gl function.
We could use them for both rendering into and for sampling from. I don't know if they can be efficiently implemented though.

l_belev
09-10-2012, 03:14 AM
@Alfonse: I dont think adding "viewable renderbuffers" would complicate the wgl or any other gl-glue-api. For example it can be done this way:
We already have mechanism to create gl contexts with attributes. Then we can define a new attribute that means "this context does not use the default framebuffer". Such context is still activated with e.g. wglMakeCurrent, but the HDC parameter is just a dummy window to make the old apis happy (it could be the same dummy window that we need to get pointer to wglCreateContextAttribsARB). Using dummys is nothing new to us, is it? You get the idea.

Alfonse Reinheart
09-10-2012, 10:58 AM
I highly doubt something like that could even work. After all, the implementation needs to know the format of the image to be displayed. By doing what you suggest, you could display an sRGB buffer one frame, then a non-sRGB buffer the next. Is that even something hardware can handle?

l_belev
09-11-2012, 09:16 AM
Why are you worried about the possibility of changing the display format every frame? After all during single frame the application itself can switch the colorbuffer format by using FBOs and/or do format converting blits as many times as it wants.
One more such would hardly be a problem. AFAIK all GPUs have flexible blitters that can convert most uncompressed formats on the fly at no higher cost than a simple copy operation.

Alfonse Reinheart
09-11-2012, 11:23 AM
Why are you worried about the possibility of changing the display format every frame?

It's not the "display format" that's the problem. It's the display.

If you want to display an sRGB framebuffer, currently you must create an sRGB default framebuffer. The fact that the default framebuffer is sRGB is an intrinsic, inflexible part of the default framebuffer.

What you're suggesting assumes that the "display" has no format. That an sRGB default framebuffer is ultimately no different from a linear RGB default framebuffer.

There's a difference between "what you render to" and "what you show in a window." Unless there is evidence that "what you show in a window" is as flexible as "what you render to," I prefer to err on the side of getting the most useful functionality with minimal disruption of the current system.


AFAIK all GPUs have flexible blitters that can convert most uncompressed formats on the fly at no higher cost than a simple copy operation.

I don't want to do a copy operation. And with SwapBuffers, you don't have to. Why should we accept that (likely minor) performance penalty even when we don't use it? Why not get the best of both worlds: the ability to talk directly to default framebuffer images, and the ability to get the fastest buffer swapping performance possible.

kRogue
09-11-2012, 01:59 PM
I don't want to do a copy operation. And with SwapBuffers, you don't have to. Why should we accept that (likely minor) performance penalty even when we don't use it? Why not get the best of both worlds: the ability to talk directly to default framebuffer images, and the ability to get the fastest buffer swapping performance possible.


Um, err... under X11 with a compositing window manager the situation is quite, quite ugly. For X11 the incantantation is this, and I joke you not:

Create a pixmap from the window handle (this may or may not be a copy operation, depends)
Create a GL texture from the pixmap (this is not a copy usually, usually it is texture from pixmap)
draw the GL texture


This sequence is executed _each_ freaking frame, not just startup or when a window is made, but each and every freaking frame. In theory an optimized X11 system may make it so that the "pixmap" is not created each time, but such a scheme either requires that the application is blocked by the compositor or that somehow some triple buffering is going on.... often enough, the contents of the window's front buffer might be copied to the pixmap.... one can argue that vsync means blocked by the compositor but not really, it is messier than that. On a related note, X11/EGL is a cesspool, as EGL you set the color buffer bits, and X11 window does too.... so crazy the world is that for most paltforms if the bits don't match, you get an EGL error, but there was an extension for Tegra that had it convert that which you renderer to whatever format the window was, the extension was EGL_NV_post_convert_rounding (you can see it in use for example in Qt code: http://qt.gitorious.org/qt/qt/blobs/4.7/src/gui/egl/qegl_x11.cpp )

Fullscreen is a different matter, but under X11, a window is fullscreen or not is actually really a message/flag the compositor sees and decides, at it's leisure, to obey.

But back to l_belev(not Illian :p) on paper the window's framebuffer could change all the time _IF_ the machinery of pixmap and texture from pixmap took into account various color formats.. but AFAIK, all that is really available is RGB565, RGBA8 and RGB8 for X11 pixmaps....

Other windowing systems around are Android's, iOS, Windows... iOS already does the render buffer thing, but it means that an application is NOT double buffered, rather there is an iOS call that says "present the renderbuffer [possibly at vsync]". Android uses EGL, but I don't remember/know what the Android platform does with respect to windowed stuff ... for fullscreen, it likely is just simple double buffering....