Mipmap levels and video memory

[Ysaneya]

I think it's common knowledge nowadays that all consumer cards store the whole mipmapping chain in video memory as soon as the texture is used at all. In other words, if you've got a 4096x4096x4 mipmapped texture and only an area of 64x64 pixels is visible on screen, the whole 80 Mb of data will be uploaded into video memory, even though the rasterizer will only need to access 16 Kb of these. I know in practise there is other considerations (trilinear filtering needs access to the closest mipmap chains), but why the hell can't the drivers manage this memory better ? The amount of textures you could have in your scenes would increase tremendously, and even performance should increase (no need to upload all this data from system memory if it's not yet in video memory). So is there a specific reason why this isn't possible ?

Y.

[Won]

Well, it is possible and 3Dlabs cards already do this. They perform demand paging of texture data and view video memory as a "cache" of AGP memory. Future commercial (ATI/NVIDIA) cards might do this soon, too. It'll be more worthwhile when superbuffers factors most memory operations to orthogonal operations. Carmack suggested it a while ago (add TLB to GPU) but I think that's what the AGP GART actually is. With PCI-express cards, it'll be slightly different.

-Won

[dorbie]

Yes there's a reason drivers can't manage this under some circumstances.

At any instant your code could generate fragments that requires that high resolution image. OpenGL has no idea when or if this might happen. You made that texture, hopefully for a reason. The performance penalty for this could be severe and defeat any potential optimization.

About the only thing you could do is perhaps use MIN LOD to give the driver a chance to ignore the higher resolution if you don't need it. Some implementations may consider it, I had a lengthy debate with a driver developer at NVIDIA over this some time ago now, he was basically saying that they ignore this and it's a bad idea, maybe he's right (don't think so for smart developers). There are things that implementations can do to manage texture memory (and some do AFAIK) like storing tiled fragment groups and paging them when not resident etc I don't think they like discussing this kind of stuff because it's proprietary and some of it's patented. It's always short of the potential for applications to be smart about paging requirements, but if the drivers ignore the few mechanisms we have to hint at memory priorities it is a bit of a downer.


[This message has been edited by dorbie (edited 02-06-2004).]

[Ysaneya]

I agree there might be some tricky situations where the full mipmap hierarchy is needed, but i'm sure a smart driver could detect it, and you must admit that's not something you do for 99% of your textures. Unless i'm missing something, it could be implemented with the current generation of graphics card, or even older. I mean, the tendency today is to add more and more transistors, to duplicate the onboard memory, etc.. and to me it just seems that field of optimization is left abandonned. Wouldn't you like to "automagically" multiply by 10 the amount of textures you can use in your scenes ?

Y.

[Korval]

but i'm sure a smart driver could detect it, and you must admit that's not something you do for 99% of your textures.

Glslang gives a fragment program the ability to compute the fragment's LOD bias on a per-fragment basis. This, pretty much, kills any hope of allowing the driver to have any clue as to whether or not any particular mip LOD will be used.

The video-memory-as-cache mechanism could be quite useful, and I'm surprised that major vendors don't implement it. It would have to be done in hardware, as a software version of this mechanism would rely on the hardware firing some kind of interrupt to the driver, and therefore potentially stalling more than it needs to.

But, perhaps card makers feel that this mechanism would actually be slower. Maybe they've done some hardware tests, and found that it isn't as good as one mught think.

[Henri]

I think this is something that might make it into the next DirectX standard; the one witch will be released along with Longhorn.
There have been some discussions about using a virtual memory system. The effect would be that we can use video memory, AGP memory and system memory in the exact same way. And then the driver will manage the textures and vertex data or anything you throw at it, in the best possible way.

This is similar to what we have when dealing with system memory. You can’t really tell when you use the ram memory or the page file... Now I know you guys are OpenGL fans, but if it makes it into the standard of DirectX then the developer have to make sure that the hardware is not the limiting factor. So then it can’t be too hard to get it into OpengGL as well! And that’s nice. Instead of using 80mb chunks, the texture can be split by the memory subsystem to pieces of maybe 4 kbyte… If you’re interested take a look at this:

Beyond3D article

[Ysaneya]

Glslang gives a fragment program the ability to compute the fragment's LOD bias on a per-fragment basis. This, pretty much, kills any hope of allowing the driver to have any clue as to whether or not any particular mip LOD will be used.

But again, how often do you think that feature will be used ? The driver could easily parse the shader to find the mipmap bias instruction, and then flag the bound texture as needing all the mipmaps.

Y.

[Enbar]

Ysaneya, even if you exclude glslang the only way I can see the driver handling the easier cases is by doing software vertex transforms to calculate eye z values. This would be prohibitively expensive.

[idr]

There are basically only two ways that the driver could do it all by itself. One is the 3dlabs way, which has already been mentioned. What I didn't see mentioned is the fact that 3dlabs has at least one patent on the technology. I suspect that's why no other vendor has used that technique.

The other would be for the driver to calculate lamda value for every polygon using a given texture. In order to do that, not only would the driver have to transform and clip *all* of the polygons, but it would practically have to rasterize the polygons (to compute the min/max rho values).

Do you hear that flushing sound? It's performance going bye-bye...

The only reason I know this is because I just finished looking at doing something like this in the open-source 3D drivers for XFree86. As a side note, there is a 3rd way to tackle some of this, but it requires a little hardware support and a little application support. It's called clipmaps. Check out the SGI extension spec and the original paper on the subject. It's a pretty clever idea.

[This message has been edited by idr (edited 02-06-2004).]

[This message has been edited by idr (edited 02-06-2004).]

[OpenGL guy]

Originally posted by Ysaneya:
But again, how often do you think that feature will be used ? The driver could easily parse the shader to find the mipmap bias instruction, and then flag the bound texture as needing all the mipmaps.

Even without using GLslang, the driver has no idea what the location of the transformed vertices or computed texture coordinates will be. A triangle may be small, but if the texture coordinates are close together, then you may need a higher resolution mipmap anyway.

If the driver has to compute vertex positions and texture coordinates, then you might as well be doing software vertex processing.