Official OpenGL mechanism for vertex/pixel "shaders"

nVidia and MS worked together on creating DX8 with its vertex and pixel “shader” architecture. This is new functionality that GL lacks. A GL extension for vertex programs was added by nVidia, and I expect that when NV20 comes out they will add an extension to access its per-pixel abilities (dependent texture reads and the like). This is, of course, good. We wouldn’t want new hardware to come out without us being able to access it’s features through GL.

In the long term, though, this kind of programmable pipeline will become more and more prevalent. I believe that eventually this kind of functionality will have to be folded into the official OpenGL specification. Either that or this is the time for an official break of a games-specific flavor of GL from the existing system. At this point taking advantage of vertex arrays, multitexture, complex blend modes and shading operations, and programmable per-vertex math leaves one writing code consisting only of extensions, it seems. This functionality should either be brought into GL proper, or should be spun off into a seperately evolving spec that consumer games cards would implement (a “gaming” subset, similar to the “imaging” subset).

Some questions:
Is the DX8 API for a programmable pipeline (and the corresponding shader language they have chosen) the “right” choice? What should happen in the next release/s of OpenGL to adapt it to the reality of T&L hardware and programmable GPUs?

You can be absolutely confident that not only will all the DX8 features be exposed in OpenGL, but that we will in fact provide more and better features in OpenGL.

If you look at what DX8 provides today, almost everything that is truly “new” in it is still being emulated in SW. So, at this point, it really doesn’t offer anything fundamentally new. Many of the features in it have been around in OpenGL for a long time. Vertex streams look a lot like vertex arrays. 3D textures are in OpenGL 1.2. Pixel shaders are actually less powerful than register combiners. And so on.

Specifically, on the topic of vertex programs, we do feel that the API chosen by DX8 and in NV_vertex_program is the right API for this functionality.

The ARB is currently looking at programmable geometry. I can’t comment further on the activities of the ARB, for a variety of reasons.

  • Matt

Matt, a few related questions if I may:

  1. NV were first off the block with OpenGL vertex programs, and were very influential in the design of DX8 - certainly far more so than any other vendor. Is this relatively vendor-specific design choice likely to hamper ARB standardization in this area?

  2. Most of the recent GL extensions have focused on pipeline programmability, which is very low-level compared to the rest of OpenGL. Are there any efforts underway to balance this trend by providing simplified access to common applications of this programmability? Requiring programmers to reimplement the entire T&L pipeline (using a whole new pseudo-assembler language) to use new effects makes for great demos and cutting-edge games, but probably isn’t going to win many converts among more mainstream users.

  3. If and when the ARB does standardize on a programmable-pipeline scheme, can we assume that the pseudo-ASM language will also be standardized?

  4. If you can’t comment on ARB progress, can you comment on why you can’t comment on ARB progress? No meeting minutes have been published for quite a while now. Is it a case of IP worries (shadow of the Rambus fiasco), or what?

  5. We’ve heard nothing for AGES about official MS support for 1.2, which implies that we can expect official support for a putative 1.3 sometime after the heat death of the Universe. Is there anything that can be done to bypass this MS bottleneck and get working (statically-linked) support for 1.2 and future versions?

I know some (all?) of these are fairly political, and I really don’t want to put you on the spot in any way - if you can’t or would prefer not to answer, that’s fine. I think everyone on this board appreciates the effort you put in to keep us up to date. I’m just curious, and figured it couldn’t hurt to ask.

Pixel shaders are actually less powerful than register combiners.

Really?

Why would anybody want to make a “new” feature that is less powerful than something that has already been around for a while?

In what ways can register combiners outdo the DX8 pixel shaders?

j

Pixel shaders vs. register combiners: pixel shaders are missing the signed range, the register combiners range remappings, an equivalent to the programmable final combiner, an AB+CD operation, and the mux operation. The “full” pixel shader spec has some extra features, but they are not supported by any hardware available today.

I don’t see the lack of MS 1.2 support as being much of an issue. It’s accessible as an extension and static linking would create compatibility issues (what happens if an OGL 1.2 app runs on a system with a 1.1 driver? you’ll get an obscure error message, most likely). There is talk at the ARB of a WGL “replacement”, but I think this would cause far more problems than it would fix.

I can’t discuss anything related to the ARB discussion of programmable geometry. Far too many IP issues.

It’s disappointing that the ARB hasn’t posted meeting notes publicly lately, yes. I don’t know what the deal is with this.

On the topic of low-level vs. high-level APIs, I firmly believe that we have been designing specs at “about the right level” of abstraction. On one hand we have people crying out for direct access to every single thing in the HW. On the other, we don’t want to create legacy-nightmare extensions, and we need to make the features reasonably usable. Some specs are lower-level than others, but only in the places where we believe it’s necessary – and even then, we are often still providing significant abstraction from the underlying HW.

It’s true that extensions will always add more API complexity. This is unavoidable. 3D will get more complicated, no matter what we do. The solution, I think, is that there will need to be more layers of API abstraction. You’ll probably see more 3rd-party 3D libraries or engines where someone who specializes in OpenGL or D3D has already done this work. Clearly, the solution is not to add a glLoad3DStudioModelAndDisplayItWithBumpmaps command… but someone else can provide that, if that’s what people want.

  • Matt

Matt -

Just one thing. Register combiners may be more powerful than pixel shaders, but they still don’t expose the dependent texture read functionality - and I assume upcoming nVidia HW will support that too. Is there an existing extension that exposes dependent texture reads at the right level of abstraction?

Oh, and just one more “just one more thing.” This is off the topic of the original post here but will the next-gen cards from NV that support 3D textures allow those textures to be paletted? I bought a Radeon for a research project I’m doing that required that functionality and then found out that ATI doesn’t believe in paletted textures.

Dependent texture reads are in DX8, and so we’ll have them in OpenGL too.

3D textures: When we support 3D textures, we’ll definitely also support paletted 3D textures. I don’t know if the Radeon HW actually supports paletted textures at all.

Now, on a slightly related subject, when ATI put up their DX8 devrel material, I noticed something interesting about 3D textures on Radeon…
http://www.ati.com/na/pages/resource_centre/dev_rel/sdk/RadeonSDK/Html/Info/DirectX8.html

And I quote:

The RADEON™ does not support multiresolution 3D textures (i.e. volume mip maps) or quadrilinear filtering.

Interesting that they haven’t really felt much need to mention this up until recently.

  • Matt

Yes, the Radeon seems to have paid a price in flexibility for being the first out of the gates with volume texturing.

I think the reason they can’t do volume mipmaps probably has to do wtih the amount of filtering that would be involved in implementing MIPMAP_LINEAR for volumes. If I remember correctly the Radeon 3-texture pipeline is limited by the number of linear filters it can do. It can handle bilinear filtering on three textures, but if you turn on trilinear filtering for even one, then you can only do two simultaneous textures (albeit with trilinear on both). Since a volume texture already uses trilinear for regular sampling (wheras a 2D texture uses it only for MIPMAP_LINEAR) I think that mipmap interpolation for even a single volume texture would go over their limit of six directions of linear interpolation. It seems that it would be best to have the texture units be truly orthogonal, so that the filtering in each may be chosen freely.

Fortunately, for my project, volume mipmapping is not required. Unfortunately paletted textures are absolutely critical…

Yes, true LINEAR_MIPMAP_LINEAR support requires quadrilinear filtering. But plain old LINEAR filtering is trilinear with 3D textures, so I think the lack of mipmap support may not be related to filtering concerns.

ATI also doesn’t advertise their trilinear restriction very openly… I believe they use two texture units to do trilinear, so you can do 1 trilinear and 1 bilinear but not 2 trilinear, and if this is correct, it actually constitutes a small cheat on their part – similar to how the V5-5500 has been bashed in some circles for supporting only bilinear in combination w/ multitexture, causing its benchmarks to be slightly overstated.

Bringing you your daily dose of FUD,

  • Matt

Upon talks about card weaknesses, when will we see a nVidia card with near-Matrox image quality? Radeon is almost there, V5 not too far behind.

I don’t know what the deal is with all the various claims of 2D image quality I’ve seen.

I could rant for hours on how people on the web seem to know absolutely nothing about image quality… (not that the average person, or even the above average person, on the average HW web site seems to know anything about 3D either, no matter how much 31337 insider information they claim to have)

For one, 2D and 3D image quality need to be clearly separated. 3D image quality is the mapping from OpenGL/D3D commands to pixels in the framebuffer and can be measured objectively, while 2D image quality is the mapping from the pixels in the framebuffer to the image on the screen, and is generally very subjective.

In terms of 3D image quality, ever since the TNT, I think it’s safe to say that we’ve been right up there at the top and that we’ve only improved since then.

For 2D image quality, I’ve heard so many contradictory stories that I don’t know who to trust any more. I’ve heard everything from “my old Mystique II has better 2D image quality than my GeForce2” to “G400 is underrated, TNT2 was better and GF kills it.” Since 2D image quality is a function of so many factors (video card, monitor, cable to monitor, resolution, color depth, refresh rate), and it’s analog, I have a feeling that the confusion stems from people failing to do reasonable comparisons. Comparing video card X on monitor A with video card Y on monitor B tells you absolutely nothing about X, Y, A, or B whatsoever!

I’ve also seen all sorts of claims about how one card has brighter colors or somesuch, and so it’s better. These are equally ridiculous. It’s called “gamma”, folks. If gamma isn’t sufficient, well, with the number of controls on the average monitor and in our control panel put together, you have absolutely no excuse to be complaining about the colors.

In the long run, I would imagine that the 2D image quality issue for video cards will go away entirely. Digital connectors will probably do the trick.

  • Matt

What I was mainly thinking about was the generally blurry output on high resolutions on most cards based on nVidia chipset. The difference between a G400 and a GeForce is rather big. Radeon is only slightly behind, while the GF’s is in their own class below. Sure, many people don’t have a problem with this, but this is one reason I have still never bought a nVidia product. The other reason is that nVidias webpage plainly sux. It only tells you how fantastic their card are, but all the marketing BS gets boring rather quickly. When one starts to wonder where the techical details about the card can be found you’ll only find that it’s not there. At least I’ve never found any kinds of feature list of the cards. Amazing that you must read reviews on the net to find those things out and not being able to find it at their own homepage.

I’m not saying you’re wrong about 2D image quality, but what I’m saying is that I’ve heard so many contradictory claims on the subject that I don’t know who to believe any more.

I personally have never had a problem with our 2D quality. It’s also not my department; the OpenGL driver has absolutely no impact on 2D image quality.

And I’ve talked to several people in person who say that the whole issue is very exaggerated and that they have tried G400 and Radeon and saw no difference whatsoever at any resolution.

I’ve also seen claims that many of the problems come from board vendors using a very cheap filter (i.e. components are not up to spec) on the VGA signal. I don’t know if this is true.

  • Matt

Oh, and as for our web page, I was able to find a list of features in a few minutes:
http://www.nvidia.com/Products/GeForce2Go.nsf/features.html

And if you’re expecting “the dirt” on our web page rather than info from marketing, well, I think it’s safe to say that that isn’t going to happen – the whole point is to present a positive outlook. If you want more technical info, that’s what the developer pages are for. If that’s not good enough, then either we (1) only have so much time or (2) decided that it is in our best interest to not publish that information. This is how all companies work, not just us…

I think it’s also safe to say, from looking at our competitors’ web pages, that they follow a fairly similar policy. Marketing info on the main pages, technical info on the developer pages, and only the info that they feel is fit to post.

  • Matt

Sure, most people don’t have too sensitive eyes. But I’ve seen various nVidia cards in action. None have been satisfying in higher resolution. It may perhaps vary between vendors, but I’ve heard from many sources that the reference design (which most vendor build their cards from) is the problem.

The web page thing. That link wasn’t exactly what I was looking for. I mean, that feature list isn’t even half the length of the feature list of a V5. If I wonder if the GTS supports say anitsotropic filtering and to what degree. Where should I find that information?
It only tells you that the card are go damn great, but never tells you in what aspects. It’s just the latest and greatest. 95% of the page i marketing BS, the other 5% is useful information.

Look at this exemplary ATi side: http://www.ati.com/na/pages/technology/hardware/radeon/techspecs.html
It’s easily found on their site.
3dfx has a good and complete feature list, just aswell as Matrox.

Okay, the vendor-bashing here is far more off-topic than even my OT post. If you don’t like nVidia cards, that’s you own issue and nobody will make you buy one. The fact that the GeForce is pretty much the defacto standard at the moment means that a few people out there think nVidia cards are okay. Besides, issues like that don’t belong on a developer forum they belong on a gaming site.

Back to the original topic (sort of).

It seems to me that with programmers demanding more and more flexibility with vertex and pixel shaders, the industry is going to end up having pixel and vertex shaders with an almost unlimited number of instructions, texture fetches whenever the programmer wants from any texture unit, dozens of different machine instructions, their own set of registers to store data, and so on. Pixel shaders will end up doing almost any sort of calculation.

Sort of like a CPU does.

It seems that in a couple years, “all” a graphics card will be is a completely programmable hardware T&L unit, and a couple dozen general purpose pixel pipelines.

Would this simplify graphics chip designs? Instead of having to fit many different types of functionality on the chip, the chip could do it using the pixel shading programming language.

I find it sort of strange that graphics chips, which were diverging from CPU’s in terms of special functions and such, might be evolving to become more like a CPU.

What do you think?

j

>web seem to know absolutely nothing about
>image quality

laughs
it’s funny cause it’s true

yeah, a lot of ppl think pixel shaders are
holy grail, like i was talking on the phone in this game dev interview, and the guy was telling me pixel shaders are the bomb, etc…
haha,

dude didn’t know much about them.
and the fact that there still is a bunch of time till they are supported in hardware, makes them totally useless to use… software is horribly slow.

yes combiners have a buttload of functionlity and for demo making they are the best way to take.
but many people can’t really use OpenGL in production.

the reason?
cause some companies spit out **** drivers.
and thats’ the bottom line.

imho Nvidia’s version of OpenGL wipes the floor on the current hardware against *D3D, But the sad part is.
it only works on nvidia cards.
heh

*hoppe added some really good code into the DxRelease build/d3dx and i’m pretty sure a lot of ppl like the progresive meshing…

and then the other fact, that directX is backed by a company with lots of money.
and with D3D it’s really s “Standard”,

like i blasted mark an email cause i was really ticked off cause of all the ip/nvidia crap on the spec and check out the reply i got…

problems/urls… http://www.angelfire.com/ab3/nobody/email_rebut.html

mark’s reply http://www.angelfire.com/ab3/nobody/kilgard_rebut.html

and we are trying to standarized OpenGL right?
food for thought.

laterz,
akbar A.

I think that in the long term OpenGL is going to be on a slow road to obsolesence. The simple reason is that purely raster APIs like OpenGL existed in order to be a lowest common denominator that almost all hardware could support. The GL API is huge and often unnecesarily so, it seems. MS had the right idea with DX8, they introduced a far simplified model of the API. All vertex data is stored in Vertex Buffers, all transformations are encoded as Vertex Shaders and all shading is encoded as Pixel Shaders. In the best case all the data remains resident on the card, and the app only signals when to switch vertex data, textures or shaders. Not using calls that directly rasterize primitives gives huge speedup.

A truly modern API must embrace this style of programming, but OpenGL, I think, ends up bringing too much of its legacy along too. It will be interesting to see how GL weathers the introduction and wide acceptance of vertex and pixel shaders, and how it will evolve as rasterization APIs eventually give way to scene-graph APIs.

Originally posted by timfoleysama:
Okay, the vendor-bashing here is far more off-topic than even my OT post. If you don’t like nVidia cards, that’s you own issue and nobody will make you buy one. The fact that the GeForce is pretty much the defacto standard at the moment means that a few people out there think nVidia cards are okay. Besides, issues like that don’t belong on a developer forum they belong on a gaming site.

So, it’s OK that mcraighead out of nothing gets the weaknesses of both Radeon and V5 into the discussion, but I better not talk about the GeForces weaknesses?
Sure, it’s OT, but that was the filtering limitations of the V5 and Radeon too.

I was not bashing nVidia and I haven’t said GF is a bad card. I said that the image quality didn’t satisfy me and I asked when we would see this problem solved.
Also, image quality and feature belongs IMO more to developer forum than a gaming site. The average developer care much more about those two issues than the average gamer.