PDA

View Full Version : questions about OpenGL with multilpe threads



cournia
12-03-2010, 03:35 PM
I'm starting to explore the use of multiple threads communicating with
the OpenGL driver. Several questions have come up that I hope you can
answer. As background, my application makes heavy use of OpenGL based
GPGPU and OpenCL to perform computations. My goal is to have these
computations performed in the background in separate threads, while
having my GUI remain responsive to the user. Additionally, my main
GUI thread also uses OpenGL to perform various visualizations. I'm
developing on Linux with an NVIDIA GPU.

On to the questions:

1. Given an application with multiple threads, each thread with its
own context, and each thread actively sending commands to its
context, what is the execution order of these commands across
contexts:

* Do all of the commands in a context execute, then the GPU moves
to the next context?

* Is some sort of time-slicing done between contexts? For example,
if we have 2 contexts, each with 200 built-up commands, does the
GPU run 20 commands from context 1, then 20 commands from context
2, then 20 more commands from context 1, etc?

2. If I have an application with 48 threads, each of which wants to
communicate with OpenGL, should I:

* Have 48 GL contexts?

* Have 2 contexts? One for the main GUI thread, and another for
computation. In this second thread, it would be up to me to
serialize all OpenGL commands to the context.

* Have a single context? If so, how do I keep the GUI thread
responsive if I'm performing long running OpenGL GPGPU tasks in
the same thread?

3. If my application has 16 contexts, and the GPU is currently
processing a single command in the current context that takes 60
seconds to complete, will the GPU wait until the long running
command completes before switching to another context, or will the
GPU recognize that the command is taking a long time, pause the
command, and switch to another context in a effort to keep the
system responsive?

4. When using a thread/context pair to load texture data; after a call
to glTexImage, will the texture data be immediately available to
other shared contexts? If not, should I use ARB_sync or some other
method for synchronization? If I use PBO's to upload the texture,
does the answer to this question change?

5. If I create a shader in a thread/context pair, is it immediately
available in other shared contexts?

6. If I upload a uniform to a shader, is it immediately available in
other shared contexts?

7. Are shader uniforms and texture state (such as a texture's wrap
mode) shared across contexts if the contexts are shared?

8. Is it possible to cancel the rendering requests for a context.

Any help is greatly appreciated. Thank you for your time.

Alfonse Reinheart
12-03-2010, 05:08 PM
what is the execution order of these commands across
contexts:

There isn't one. It is up to you to synchronize all of your access to OpenGL. Or, to put it another way, OpenGL makes no multithreading guarantees whatsoever. None of the functions are guaranteed to be re-entrant with themselves or any other OpenGL function. They may be, but OpenGL makes no guarantee of this.

There are some re-entrant things that are generally considered "safe", such as changing the state of objects that are not actively being used by other threads. But currently, OpenGL makes no guarantee of any of this.


If I have an application with 48 threads, each of which wants to communicate with OpenGL

I know this is hypothetical, but you really should not do that. Even if you had a 48-core processor, synchronizing access to GL across 48 threads is probably not helpful.


When using a thread/context pair to load texture data; after a call to glTexImage, will the texture data be immediately available to other shared contexts?

What do you mean by "immediately available?" Assuming synchronized access (see previous point), it will be just as "immediately available" as if you called the function from that context. The object is shared between the contexts, and the function in question changes the state of the object.


Are shader uniforms and texture state (such as a texture's wrap mode) shared across contexts if the contexts are shared?

What would be the point of sharing objects if the contents of those objects are not shared?


Is it possible to cancel the rendering requests for a context.

No. Once you issue a rendering call, it's done. It may not have completed yet, but it is on the GPU's todo list.

Kurt Hudson
01-02-2011, 04:59 PM
It is up to you to synchronize all of your access to OpenGL. Or, to put it another way, OpenGL makes no multithreading guarantees whatsoever. None of the functions are guaranteed to be re-entrant with themselves or any other OpenGL function. They may be, but OpenGL makes no guarantee of this.

There are some re-entrant things that are generally considered "safe", such as changing the state of objects that are not actively being used by other threads. But currently, OpenGL makes no guarantee of any of this.


Do you mean that commands issued for different context may result in errors? Indeed, specs say nothing about such "multithreading safe guaranties", but AFAIK client-server model makes OpenGL obliged to support multiple clients, doesn't it?

Alfonse Reinheart
01-02-2011, 05:36 PM
but AFAIK client-server model makes OpenGL obliged to support multiple clients, doesn't it?

That doesn't mean an implementation can serve them both at the same time. Only that it can serve multiple clients.

And of course, there is the question of having shared objects. If two contexts do not share objects, then there is no multithreading issue (since the per-context state is unique to each). The issue is race conditions with regard to the state of objects.

Kurt Hudson
01-03-2011, 06:02 AM
Sorry, I'm confused by all this and don't understand :). There are race conditions in OpenGL and user have to implement a synchronization to protect inner OpenGL objects?
In spec (4.1 core) I find nothing like "OpenGL commands related to shared object are unsafe", please point on it if I'm wrong. Also, it says nothing about OpenGL thread unsafeness (except one thread per context rule).
Does all this means that race conditions are only a lack of implementation quality?

Alfonse Reinheart
01-03-2011, 09:55 AM
In spec (4.1 core) I find nothing like "OpenGL commands related to shared object are unsafe", please point on it if I'm wrong. Also, it says nothing about OpenGL thread unsafeness (except one thread per context rule).

The lack of that line does not mean that they are safe. Indeed, the lack of any discussion of threading consequences means that the OpenGL spec does not even consider threading consequences. Hence why OpenGL is not thread-safe.

Thread safety is not something you assume to be true unless you are told otherwise. Thread safety is something you must specify, and until it is specified, it must be assumed to not be true.

mbentrup
01-03-2011, 10:45 AM
1. Given an application with multiple threads, each thread with its
own context, and each thread actively sending commands to its
context, what is the execution order of these commands across
contexts:

* Do all of the commands in a context execute, then the GPU moves
to the next context?

* Is some sort of time-slicing done between contexts? For example,
if we have 2 contexts, each with 200 built-up commands, does the
GPU run 20 commands from context 1, then 20 commands from context
2, then 20 more commands from context 1, etc?


The order of the commands is undefined, the only the order of commands within the same context is guaranteed. (The only exception are Sync-Objects which may be queried from any context in the current share list.)



3. If my application has 16 contexts, and the GPU is currently
processing a single command in the current context that takes 60
seconds to complete, will the GPU wait until the long running
command completes before switching to another context, or will the
GPU recognize that the command is taking a long time, pause the
command, and switch to another context in a effort to keep the
system responsive?


Also undefined. I'm pretty sure that drivers will not stop running commands, but the latest generation of GPUs can run several programs in parallel, so the large command may not block all other contexts (if you are lucky - again no guarantees).



4. When using a thread/context pair to load texture data; after a call
to glTexImage, will the texture data be immediately available to
other shared contexts? If not, should I use ARB_sync or some other
method for synchronization? If I use PBO's to upload the texture,
does the answer to this question change?


See Appendix D of the GL Spec:

"If the contents of an object T are changed in a context other than the current context, T must be attached or re-attached to at least one binding point in the current context in order to guarantee that the new contents of T are visible in the current context."

Of course this has to happen after the state-changing command has been processed by the GL on the other context. The GL function calls usually only fill the command queue, so you'll have to wait for a Sync object to check that the command has finished and then rebind the texture/buffer object etc. to get the new contents.



5. If I create a shader in a thread/context pair, is it immediately
available in other shared contexts?

6. If I upload a uniform to a shader, is it immediately available in
other shared contexts?


See above: it should be available once the relevant commands have been processed and you rebind the object in the current context.



7. Are shader uniforms and texture state (such as a texture's wrap
mode) shared across contexts if the contexts are shared?


I'm not sure, but I think yes.



8. Is it possible to cancel the rendering requests for a context.


no.

Kurt Hudson
01-03-2011, 11:08 AM
The lack of that line does not mean that they are safe. Indeed, the lack of any discussion of threading consequences means that the OpenGL spec does not even consider threading consequences. Hence why OpenGL is not thread-safe.
Ok, it seems I get your point. OpenGL is OS independent and doesn't care about thread safety, because it "now" nothing about threads. But it now! Threads and threading related troubles are mentioned in specs many times, but there is nothing about race conditions in OpenGL calls.



Thread safety is not something you assume to be true unless you are told otherwise. Thread safety is something you must specify, and until it is specified, it must be assumed to not be true.
So, I must not expect OpenGL operation to be robust in general, with exceptions described in specs?

Kurt Hudson
01-07-2011, 04:10 PM
Finally I find it out, this issue is described explicitly, but in EGL spec, not OpenGL. Strange..


OpenGL is not thread-safe
OpenGL must be thread safe, EGL spec says:

EGL and its client APIs must be threadsafe.

One using shared contexts indeed have to take a care about synchronization.

If a buffer object is bound to more than one
context, then it is up to the programmer to ensure that the contents of the object
are not being changed via one context while another context is using the buffer
object for rendering. The results of changing a buffer object while another context
is using it are undefined.

OpenGL and OpenGL ES makes no attempt to synchronize access to texture
objects. If a texture object is bound to more than one context, then it is up to the
programmer to ensure that the contents of the object are not being changed via one
context while another context is using the texture object for rendering. The results
of changing a texture object while another context is using it are undefined.

But still no words about other objects,that may be shared according to OpenGL 4.1 core spec:

Objects that can be shared between contexts include pixel and vertex buffer ob-
jects, program and shader objects, renderbuffer objects, sync objects, and texture
objects (except for the texture objects named zero).

Alfonse Reinheart
01-07-2011, 04:35 PM
OpenGL must be thread safe, EGL spec says:

OpenGL is not a client of EGL. OpenGL ES is, or can be, but OpenGL proper is not.

Also, what does EGL mean by "thread safe"? There are many levels of thread safety.

The most basic form of thread safety is reentrancy. The ability to call into a library from two different threads. This means that, as long as one thread is not affecting something that another thread is currently doing (playing with objects that may be in use by another thread, modifying state affected by another, etc), you can call into the library.

Reentrancy is required if you want to be able to call into the library from other threads at all. Reentrancy allows you the possibility to synchronize access to objects and shared state. But it makes no guarantee about race conditions for accessing objects or shared state.

OpenGL is reentrant. But it has no other forms of thread safety.

Kurt Hudson
01-07-2011, 05:07 PM
Thanks for clarification about reentrancy, but I still don't understand how can one be sure what objects are actually affected by OpenGL call?


OpenGL is not a client of EGL. OpenGL ES is, or can be, but OpenGL proper is not.
Here is a pair of EGL spec quotes, that let me think that both are clients of EGL:

This document describes EGL, an interface between rendering APIs such as
OpenGL ES or OpenVG (referred to collectively as client APIs ) and an underlying
native platform window system.

Some client APIs , such as OpenGL and OpenVG , also support single buffered
rendering to window surfaces.

EGLBoolean eglBindAPI(EGLenum api);
api must specify one of the supported client APIs , either EGL_OPENGL_API,
EGL_OPENGL_ES_API, or EGL_OPENVG_API.
etc..

Alfonse Reinheart
01-07-2011, 08:00 PM
but I still don't understand how can one be sure what objects are actually affected by OpenGL call?

You read the OpenGL specification.

Just about every function in OpenGL does one of two things: affects context state or cause rendering to happen. Objects are defined as part of context state. You modify objects by binding them (thus making them part of the context's state) and then modifying the state that those objects now represent.

Almost every OpenGL function that does not cause rendering to happen will modify some piece of context state. The OpenGL specification is quite clear on exactly what state is modified by which OpenGL functions. And the specification is likewise very clear on what state each object encapsulates.

So it's not particularly hard to figure out what functions change what state/objects.

In general, the name of the function will tell you. glTexParameter modifies texture parameter state. glVertexAttribPointer modifies vertex array object state. There are of course some objects that don't fall under this neatly, but for the most part, it works.


Here is a pair of EGL spec quotes, that let me think that both are clients of EGL:

Until you can create an non-ES OpenGL context with EGL, that's just words on a page. And even then, they would only apply to OpenGL contexts created with EGL.

_arts_
01-08-2011, 04:11 AM
Thread safety is not something you assume to be true unless you are told otherwise. Thread safety is something you must specify, and until it is specified, it must be assumed to not be true.

Totally agree, even though, nowadays, I think any good library or program should be thread safe, or at least consider threading.
I don't say GL is not good library... I think it's just that OpenGL is client-server (client calls from cpu, server execution on gpu), so it sounds (maybe i'm wrong here ?) it's almost impossible to make OpenGL natively thread safe. No ?

kRogue
01-08-2011, 03:22 PM
Until you can create an non-ES OpenGL context with EGL, that's just words on a page. And even then, they would only apply to OpenGL contexts created with EGL.



Guess what: PowerVR SGX 545 (to be released) is to be a GL3.x part, this announcement for the drivers to be made was a made a year ago. Additionally, Qt has support for creating GL (not GL ES) contexts via EGL, I'd bet they tested it on something which has that support.

Kurt Hudson
01-09-2011, 10:11 AM
situation looks sad..

1. OpenGL threadsafeness is declared in EGL specs, with exceptions: bounded in two contexts shared textures and VB objects are unsafe to change. Functions that cause changes for those objects are not listed.
2. OpenGL spec consider threading (threading-related troubles are mentioned many times in spec), but doesn't declare thread (un)safeness.
3. Implementations are not threadsafe. Access to shared objects may cause errors, and we only can assume which functions affect state of given shared object.

Should we make a bugzilla report? May we offer at least to declare OpenGL thread unsafeness and list functions/commands/objects that programmer must consider as wanting synchronization.

Alfonse Reinheart
01-09-2011, 10:23 AM
Implementations are not threadsafe.

Again, what do you mean by "threadsafe"? There are many levels of thread safety. OpenGL provides reentrancy.

And to be honest, I wouldn't want the overhead of OpenGL providing more thread safety than that. It's a low-level library; you the user should be handling the synchronization, rather than forcing every OpenGL function to provide sync overhead even if the user doesn't need it.


and we only can assume which functions affect state of given shared object.

Well, you could assume. Or you can read the OpenGL specification.

If you don't want to actually find out what functions affect the state of what objects, that's your problem. But don't act like that information doesn't exist.

Kurt Hudson
01-09-2011, 11:16 AM
And to be honest, I wouldn't want the overhead of OpenGL providing more thread safety than that.
I don't wish it too, especially if it may reduce performance (and it probably will). My suggestion in previous post is to declare threadsafety exceptions explicitly and provide in specs information necessary to avoid threading problems.


If you don't want to actually find out what functions affect the state of what objects, that's your problem. But don't act like that information doesn't exist.
But it doesn't! Please provide an example of function for which it is documented what objects it may affect. Ideally specs must guarantee that only those objects are affected by it, and there no inner (invisible for client) objects (e.g variables) usage that may result in race trouble.


Well, you could assume. Or you can read the OpenGL specification.
You offer to determine it by function name. That's what I call "assume".

Alfonse Reinheart
01-09-2011, 12:05 PM
But it doesn't! Please provide an example of function for which it is documented what objects it may affect.

All of them. That's what the specification does. It specifies behavior.

OpenGL is a state machine. Functions either affect state or use the current state to render. The OpenGL specification defines what state exists, what functions modify what state, and how that state affects rendering.

Name an OpenGL function where it is not documented what state it affects.


Ideally specs must guarantee that only those objects are affected by it, and there no inner (invisible for client) objects (e.g variables) usage that may result in race trouble.

The specification specifies behavior. Even if the state the specification mention does not exactly match the implementation, it is up to the implementation to ensure that any such differences are not exposed to the user.

Therefore, any "invisible for client" variables must not be able to affect the results of any OpenGL functions. If they do, then the implementation has a bug in it.


You offer to determine it by function name. That's what I call "assume".

No, I was pointing out that many OpenGL functions use a naming convention that makes it clear what state it affects without having to look at the spec.

aqnuep
01-10-2011, 03:13 AM
But it doesn't! Please provide an example of function for which it is documented what objects it may affect.
As Alfonse said, all these things are documented in the OpenGL specification. Some of these issues even have dedicated sections or chapters.


My suggestion in previous post is to declare threadsafety exceptions explicitly and provide in specs information necessary to avoid threading problems.
Actually the specification also mentions this information, usually in the form of declaring certain situations to produce undefined results thus indicating the lack of synchronization.

Don't expect a single function's description in the SDK pages to contain all the interactions. These are described in the GL spec and were always defined.

Also, usually if some specific thing is not discussed in the spec you could assume that its implementation dependent. OpenGL is a specification that actually defines the requirements of an actual implementation. If a particular functionality/behavior does not have any specific wording in the spec, then it means that the specification allows freedom for the vendors on how they will implement it. Inherently, you cannot assume anything about these kind of things (e.g. thread safety or synchronization).

Kurt Hudson
01-11-2011, 12:06 AM
Sorry, still cant got what you are talking about, maybe all this looks clear only for those who now how implementation organized internally? :) Lets took an instance:
Function AttachShader, described in specs as
"To attach a shader object to a program object, use the command
void AttachShader( uint program, uint shader );
The error INVALID_OPERATION is generated if shader is already attached to program.
Shader objects may be attached to program objects before source code has
been loaded into the shader object, or before the shader object has been compiled.
Multiple shader objects of the same type may be attached to a single program
object, and a single shader object may be attached to more than one program object."
Even know nothing about how OpenGL implementation is organized I can assume that program object state is probably affected, and if its shared I should provide synced access. But what let me think that it must not touch shader object (maybe any counter inside it or something like), or pipeline object this program is binded to, or uniform buffer, this program is linked with? Would it access other state? probably not, but how can I be sure?


Also, usually if some specific thing is not discussed in the spec you could assume that its implementation dependent. OpenGL is a specification that actually defines the requirements of an actual implementation. If a particular functionality/behavior does not have any specific wording in the spec, then it means that the specification allows freedom for the vendors on how they will implement it. Inherently, you cannot assume anything about these kind of things (e.g. thread safety or synchronization).
Can we assume thread safety of each function is impl-dependent if EGL declare OpenGL thread safe?


Again, what do you mean by "threadsafe"? There are many levels of thread safety. OpenGL provides reentrancy.
Ok, not all threadsafety levels, only reentrancy. How can one determine this if threadsafety is mentioned in specs, and reentrancy doesn't? If there no way - lets file a bugreport to khronos and suggest them to define this in specs (OpenGL specs)?


PS: apropos, how could multiple shader objects of the same type be attached to a single program object? I see no way to select which one of those will be used for rendering and for uniform binding. Have I miss something?

Alfonse Reinheart
01-11-2011, 12:48 AM
Even know nothing about how OpenGL implementation is organized I can assume that program object state is probably affected, and if its shared I should provide synced access. But what let me think that it must not touch shader object (maybe any counter inside it or something like), or pipeline object this program is binded to, or uniform buffer, this program is linked with? Would it access other state? probably not, but how can I be sure?

You don't "assume that program object state is probably affected;" the OpenGL specification tells you that it is affected. You're thinking about this way too hard and at too low of a level.

OpenGL requires that the implementation of AttachShader must associate that particular program object with the given shader object. OpenGL considers this "program object state".

How the implementation stores this is irrelevant. It could be in the program object; it may have a list of pointers to shader objects. It could be in some "global" array. Maybe the implementation needs back pointers in the shader objects, so that they can refer to any owning program objects.

But none of that matters. What matters is the following, which OpenGL requires:

1: AttachShader will affect that state. It will create an association between the particular program and the particular shader object.

2: After calling AttachShader, GetAttachedShaders will return the particular shader object as one of the list of shaders attached to this program, so long as the shader object still exists and the shader object has not been detached with a call to DetachShader.

3: A call to LinkProgram will cause that compiled shader, and any others currently attached, to be linked within the program object.

Remember: OpenGL is reentrant. The reentrant guarantee only stipulates that, so long as you are calling functions that change the state of different OpenGL objects, you may call any functions on any threads you want. Once you start messing with the same OpenGL objects on two threads, then you have to synchronize access.

AttachShader uses two objects. And since it uses two objects, you must synchronize access to those two objects. Because this function does not modify OpenGL state associated with program pipeline objects, uniform buffers, textures, or anything like that, then the OpenGL state of these objects cannot be affected by calling AttachShader.

I say "OpenGL state" because I'm talking about the state defined by the OpenGL spec. Similarly "OpenGL objects" means objects defined by the OpenGL spec. Only OpenGL state can affect the OpenGL-defined contents of OpenGL objects, and only OpenGL state can affect the OpenGL-defined results of OpenGL rendering. If the implementation of AttachShader needs to communicate with objects not affected as OpenGL defines (for implementation reasons), then the implementation must not allow this to affect the outcome.

So if attaching a shader to a program requires an implementation to peek at the pipeline object(s) the program is attached to, the implementation must do so in such a way that it cannot affect the actual OpenGL state or anything of the kind. This includes reentrancy. This is an implementation detail that OpenGL does not allow to be exposed to the user.

Implementation details must remain hidden behind the abstraction. That is OpenGL's primary purpose.


how could multiple shader objects of the same type be attached to a single program object? I see no way to select which one of those will be used for rendering and for uniform binding. Have I miss something?

Yes.

GLSL's compilation model mirrors that of C. In C, you compile source code into one or more object files. These can even be bundled together into libraries.

But to build an executable program in C, you must take one or more object files and link them into a program.

The same goes for GLSL. You compile GLSL source code into shader objects, just like you compile C source code into object files. You link one or more shader objects into a program, just like you link one or more object files into a program.

There is no requirement that a shader object must have all of the code for a particular shader stage. Just as in C; not every object file must define "main".

The difference is that shaders tend to be sufficiently short that this kind of compile/link model isn't too useful. You generally don't see libraries of shader functions that you use where needed. It happens, but it's not the typical way that you make programs.

However, the functionality is certainly available. It works exactly like C. To access functions defined in one object file from another, the other object file must have function declarations. But only the declaration; not the definition. The linking step attaches those function declarations to their actual implementations, regardless of which object file they happen to come from.