Future of OpenGL/GPUs

[Jose Goruka]

I'm dissapointed towards where graphics cards are going. Specially now that larrabee is cancelled. To put it simple, why can't i do everything my rendering thread does entirely on the GPU? OpenCL and OpenGL are both heading towards being simple and flexible, but i still can't seem to do that.

So let's see.. how does the usual basic render thread looks like?

1) Frustum Culling
2) Occlusion optimizations (portals/visibility grid/etc.)
3) Batching (by wathever you have to bind)
4) Rendering

there's more stuff like shadows, postprocessing, but that's not relevant to my point.

Basically, what i mean is, doing all this entirely on the GPU (NO CPU!)

1) Frustum culling can be easily done on the GPU. The GPU is so parallel that even a brute force frustum culling of a million objects would do.

2) Occlusion optimizations would easily be solved pretty well on the GPU by just doing a near->far depth sort and querying/rendering a zpass (query visible/discard visible). GPUs can already do this pretty quickly. However, i'm not sure if this would be doable. However, grid-based visibility and portals would definitely be doable. Well, don't mind this point so much anyway.

3) Batching is pretty much just sorting, and GPUs can also do this extremely efficiently

4) I think command buffers nowadays mean you can have a pool of data that the GPU will interpret and follow. So i guess it should be able to render too. But i don't really think that works as they exist.

So nowadays, i guess the main issue would be between points 3) and 4).. the only way to render what i came up in 3) would be going to the CPU and back, which is just too slow.

So my humble question is, i think right now there's stuff i don't have in both OpenGL and OpenCL to be able to do all the rendering logic in the GPU.. so my question is more like.. why are GPUs not heading this way? Is there anything i'm missing?

[Alfonse Reinheart]

To put it simple, why can't i do everything my rendering thread does entirely on the GPU?

Because that isn't OpenGL's problem. OpenGL is a low level rendering system. If you want a system that takes care of all of this for you, OpenSceneGraph is more to your tastes.

there's more stuff like shadows, postprocessing, but that's not relevant to my point.

No. This is entirely relevant to your point. The stuff you outline is stuff that you do. Not every application does its stuff that way, and not every application needs to. Some applications need to do more stuff than you outline. Should this hypothetical OpenGL also do that stuff, even though you don't need it to?

A one-size-fits-all approach to this is inherently limiting in some way. That's why OpenSceneGraph is not used for games. This is why it is useful to have a low-level rendering system.

[Jose Goruka]

Because that isn't OpenGL's problem. OpenGL is a low level rendering system. If you want a system that takes care of all of this for you, OpenSceneGraph is more to your tastes.

I expected my post would be misread or misinterpreted, so let me clarify. This has nothing to do with OpenSceneGraph. I'm actually talking about low level rendering and operations that can greatly benefit from being on the GPU... as in, at the same level of stuff like computing a particle systems, skinning, or even rigid body solving (which is also done on the GPU nowadays). Frustum cull and batching are, undeniably, much faster on the GPU than on the CPU and both are already low level aspects of rendering. (even though they are not very usable nowadays). Also, you seem to focus on OpenGL, i'm also talking about OpenCL which _IS_ general purpose.

[Brolingstanz]

Fwiw I’d like to see things generalize more and it just can’t happen fast enough as far as I'm concerned.

Interop seems the slippery slope of contention but the prevailing north vesterlies seem to be taking us in the general direction of a Larrabee. I imagine the bean counters have to first fit the right business model to it.

[Alfonse Reinheart]

I'm actually talking about low level rendering and operations that can greatly benefit from being on the GPU... as in, at the same level of stuff like computing a particle systems, skinning, or even rigid body solving (which is also done on the GPU nowadays).

None of these are low-level. They have interactions with low-level components, but they are not as a whole low-level.

There are times when you don't want to do skinning on the GPU. There are times when you want to do GPU skinning differently (dual-quaternion, compared to matrix skinning). There are different ways to do frustum culling, depending on restrictions you can put on your camera (an orthographic game, for example, can much more easily frustum cull). There are so many ways to implement a particle system that you couldn't begin to cover them with a single system.

Even if you were to build some system that uses OpenGL and OpenCL to do these things, a one-size-fits all solution simply will not work for these components. You could make one that fits your needs, but it would not fit my needs.

And who exactly do you want to implement and maintain this code? IHVs? ATi is barely able to make GLSL work; there's no way they'd be able to take substantial parts of a videogame's rendering engine into their drivers.

Also, you seem to focus on OpenGL, i'm also talking about OpenCL which _IS_ general purpose.

OpenCL is general purpose. Which means you can use it to do whatever you want. That specifically means that it should not start taking on special-purpose features like frustum culling and such.

If someone wants to write an optimized OpenCL frustum culling library, more power to them. However, OpenCL should not come with such a library, nor should OpenCL providers be required to make and maintain such a library.

[Jose Goruka]

OpenCL is general purpose. Which means you can use it to do whatever you want. That specifically means that it should not start taking on special-purpose features like frustum culling and such.

If someone wants to write an optimized OpenCL frustum culling library, more power to them. However, OpenCL should not come with such a library, nor should OpenCL providers be required to make and maintain such a library.

I was going to reply more concisely to your post, but after reading this, i realized you didn't even understand my original post at all (or maybe i wasn't clear enough).

To make it short, i'm ranting that there is no way to generate render instructions from within the card (from either CL or GL), which is what makes implementing the features i described impossible. I'm not sure but i think the 360 can do it as the command buffer format is documented. Take a time and think about it since i believe i'm not asking something stupid.

[Simon Arbon]

Do you mean some sort of "Command Shader" that replaces display lists and lets you write OpenGL and OpenCL commands in a shader that is compiled & stored on the GPU?
Currently a series of commands are issued from a CPU program into a command buffer that is periodically flushed to the GPU for execution.
Display lists allow a command buffer to be compiled and stored on the GPU where it can be called like a subroutine, but this is still a linear sequence of instructions with no loops or branches (except NV_conditional_render extension which executes a block of commands depending on the result of an occlusion query).
If display lists are replaced with a command shader then it could include a lot more conditional logic and would remove the lag currently involved in waiting for the GPU to finish something, reading the result, deciding what to do, then sending new commands to the GPU.
It would also be a lot easier to coordinate OpenGL with physics and animation using OpenCL, where the vertex data remains in a GPU buffer and is not altered by the CPU.

[Ilian Dinev]

Making the gpu omnipotent any soon can be hard. That's why I proposed a first step:
http://www.opengl.org/discussion_boa...713#Post257713

The cpu changes global settings: blending/render-modes, tex-units, targets, etc. Parallelizing access to these triggers can't be done, but still can be put in one core of one cluster (of 8 or 16 cores) to work while fragments and vertices are computed in other clusters, and to append commands (via an interrupt-like instruction) to an internal FIFO cmd buffer.
In any case, per-primitive occlusion test results are needed first, instead of the currently available per-batch results; and a way to fetch them from within a geom-shader.

Anyway, with SM5 at the door, we can only hope and give detailed ideas/suggestions here - to be included in SM6 or as a feature of GL, that IHVs manage to implement+expose in SM4 or SM5 cards. Vague ideas like "do everything on the gpu" don't help anyone.
With the current model, generally there are 3 bottlenecks:
- getting results from the gpu (occlusion tests). Currently countered rather easily.
- too many render-calls. Countered by instancing, can be further aided by per-instance VBO/IBO offsets and per-instance primitive-count.
- each glXXX() call is 300-3000 cpu cycles. Currently countered by multithreaded drivers, texture-arrays, UBOs/etc and VAOs.

[Brolingstanz]

Can’t we just pretend everything happens on the GPU?

In the short term if you can make everything look as though it operates on the GPU you’ve gone a long ways towards abstracting and simplifying the programming model and increasing productivity, at least in theory (cf. Radpid Mind’s Sh, etc).

But then that’s just a development environment thing. Anyone (read: almost no one) can create a special purpose language and the development tools necessary to present the illusions of homogeneity to the user, even if under the hood the implementation is a 2 headed monster with festering pustules.

I could be wrong but I think a great many coders would be tickled pink with something short of the real GPU McCoy. Though the questions of what form it takes and what it targets remain, either of which could paradoxically deepen the original quagmire; and neither of which have much to do with GL (or so it would seem).

[Alfonse Reinheart]

To make it short, i'm ranting that there is no way to generate render instructions from within the card (from either CL or GL), which is what makes implementing the features i described impossible.

There is a good reason why this is impossible, at least as far as OpenGL is concerned.

OpenGL, as a specification, must maintain consistency about the order in which things are rendered. Conditional render is a way to make the rendering of one thing depend on the rendering of another. But where and when the dependent rendering will happen (if it does) is very well defined.

This is why threading OpenGL rendering to the same context is not possible. If I have started some process that will spawn rendering commands, then I issue some commands myself, which ones happen first? Is there some control over the order? How do you define an order for something like that? Can I tell when those commands have been completed?

The kind of thing you're talking about is very sketchy. Even the 360's GPU can't issue commands to itself. A shader may be able to build a command buffer sequence, but it can't tell the GPU to execute it. Only the application can do that.

- too many render-calls. Countered by instancing, can be further aided by per-instance VBO/IBO offsets and per-instance primitive-count.

Correct me if I'm wrong, but isn't a per-instance primitive count called "glMultiDraw*"?

Also, I'd be interested to know what kind of scene it is that you're rendering that can withstand the limitations on instancing/multidraw (namely, no shader parameter/texture changes).

To me, it is these kinds of changes that are the most limiting.