OGL2.0 : programmable texture filters

Wouldn’t programmable texture filters be nice? Just a simple programmable unit that could mix the texel from the current level mip map and any lower/upper levels.

Originally posted by Olive:
Wouldn’t programmable texture filters be nice? Just a simple programmable unit that could mix the texel from the current level mip map and any lower/upper levels.

You can do this in fragment shaders. Mipmaps are just regular textures. You can combine several texture together, based upon distance from eye…

Julien.

Wouldn’t that waste one texture unit per mip map level?

I think that, by applying some form of programmability to how a texture unit accesses textures, you’re getting close to the point when a software solution would be faster than a hardware one.

In a fragment shader you are allowed as many accesses as you want to a given texture, including sampling from different mip-map levels. So a programmable texture filter is redundant, as I see it.

j

I think he’s talking about user-defined filtering modes. Like, for example, making up your own version of anisotropic filtering that reads all of the texels in the highest mipmap and filters based on coverage.

That’s right. I was mostly thinking of some sort of sharpening filter.

That can be done with a fragment program. In the whitepapers they have an example fragment shader that convolves a texture to blur it. Sharpening can be done through a convolution filter as well.

j

That can be done with a fragment program. In the whitepapers they have an example fragment shader that convolves a texture to blur it. Sharpening can be done through a convolution filter as well.

And how many texture units does that burn? Therefore, if you wanted to do something in addition to that, you must resort to multipass. If you wanted to resort to multipass, you wouldn’t care what programmable features are in the card, since a reasonable set of blend modes lets you emulate virtually anything in multipass rendering.

The number of texture units burned is 1. You are allowed as many accesses into a given texture as you want with fragment shaders. This would leave plenty of room for other textures, still with the same fragment shader.

And with programmable texture filtering, the GPU would have to do multiple accesses into the texture transparently to the user to accomplish the same effect. This would seem, to me, to end up costing roughly the same as the fragment shader approach in terms of processing, but it would require adding a whole new area of functionality into OGL 2.0, which would mean more time and effort to finalize the specs, implement drivers, etc.

j

The number of texture units burned is 1. You are allowed as many accesses into a given texture as you want with fragment shaders. This would leave plenty of room for other textures, still with the same fragment shader.

Do fragment shader really guarentee that you get a virtually infinite number of accesses (up to the limit of the program) of the same texture? That’s a pretty hefty restriction on the hardware. I can see a lot of hardware that isn’t 2.0 compliant based solely on this restriction (a Radeon 8500, for example. GeForce3’s and 4’s were out of the running at being able to do it at all). This seems like a rather strict requirement.

And with programmable texture filtering, the GPU would have to do multiple accesses into the texture transparently to the user to accomplish the same effect. This would seem, to me, to end up costing roughly the same as the fragment shader approach in terms of processing, but it would require adding a whole new area of functionality into OGL 2.0, which would mean more time and effort to finalize the specs, implement drivers, etc.

However, you don’t have precious fragment shader program space taken up by new filtering options. Not only that, the location of the hardware is an issue; if you wanted to implement a better anisotropic filtering, you would need access to information that only the texture filtering unit has (for instance, the coverage of a pixel-quad in texture space, as well as the particular mipmap level[s] to use). The form of “programmable filtering” you are describing is not nearly as general purpose as the one I am describing; at best. Besides, isn’t 2.0 supposed to last for some time? If so, it has to have features that are forward looking, so that we won’t have to create an OpenGL 3.0 in 3 years.

I can see a lot of hardware that isn’t 2.0 compliant based solely on this restriction

As I understand it, OpenGL 2.0 is not meant for todays hardware. Not even next generation of hardware. You will have to wait a few generations untill you see a (near) full 2.0 compliant hardware.

Considering that the OGL 2.0 fragment shaders are written in a high-level language that’s almost identical in syntax to C, it seems silly to restrict the number of texture accesses. It’s like saying that you can only access a variable a limited number of times in a program.

However, you don’t have precious fragment shader program space taken up by new filtering options.

Well, the new filtering options would have to go somewhere. They would use up just as much space as if they were put in the fragment shader.

Not only that, the location of the hardware is an issue; if you wanted to implement a better anisotropic filtering, you would need access to information that only the texture filtering unit has (for instance, the coverage of a pixel-quad in texture space, as well as the particular mipmap level[s] to use). The form of “programmable filtering” you are describing is not nearly as general purpose as the one I am describing; at best.

If a programmer really wants to implement better anisotropic filtering in a shader setup, they can. Information about texture coordinates and triangle slope, etc. can all be calculated in a vertex shader and passed on to the fragment shader for each pixel. Voila, you have all the information you need to find out just how much texel space each fragment takes up, and you can implement your special filtering technique.

It probably won’t be as fast as a dedicated hardware setup, but cases where somebody would want to do this are probably going to be rare enough that I don’t think it justifies the extra hardware expense of programmable texture filtering functionality on chip.

j

Considering that the OGL 2.0 fragment shaders are written in a high-level language that’s almost identical in syntax to C, it seems silly to restrict the number of texture accesses. It’s like saying that you can only access a variable a limited number of times in a program.

As I understand it, OpenGL 2.0 is not meant for todays hardware. Not even next generation of hardware. You will have to wait a few generations untill you see a (near) full 2.0 compliant hardware.

In both cases, this basically means that the current situation of everybody having their own extensions won’t change at all in 2.0. If your hardware can’t handle what the fragment shader tells it it must be able to handle, then you can’t use the fragment shader functionality to implement what your hardware does support. Therefore, your only alternative is to implement your own version of fragment shaders. If the fragment shader specification didn’t require that functionality, then it wouldn’t be a problem. It may suggest this functionality, but it shouldn’t force implementations to do something just because hardware in the future will.

If there is no way to make fragment shaders work on current or near-term hardware, then there is no point in making OpenGL 2.0; it’s a waste of everybody’s time, since no one will be able to implement fragment shaders. Either that, or everyone will implement parts of fragment shaders, but they will have to give out notices that if you attempt to do X in their implementation, it will fall to software or something. Both of these are unacceptable.

Fragment shaders should be implemented as a basic subset of operations that an implementation must allow, followed by additional functionality or generality that an implementation may provide. Take matrix stacks for example. OpenGL 1.0 implementations require that the modelview matrix stack be at least 32 elements deep, but implementations can allow more.

By that token, fragment shaders should be able to say, you may access a texture up to twice, but an implementation may allow for more (or even infinite) accesses. This is not a difficult thing, and it would allow current hardware like Radeon8500 and near-term hardware (like NV30 and R300) to actually participate in OpenGL 2.0.

Well, the new filtering options would have to go somewhere. They would use up just as much space as if they were put in the fragment shader.

Yes, but they would be their own type of program with their own type of storage. Therefore, the user will not have to take up fragment shader space for things that have nothing to do with fragment shaders.

If a programmer really wants to implement better anisotropic filtering in a shader setup, they can. Information about texture coordinates and triangle slope, etc. can all be calculated in a vertex shader and passed on to the fragment shader for each pixel. Voila, you have all the information you need to find out just how much texel space each fragment takes up, and you can implement your special filtering technique.

The point of this is that 2.0 is supposed to be future looking. It is reasonable to believe that programmable texture filtering (as separate from fragment programs) is something that may happen in 3-5 years (if not sooner). It wouldn’t be that difficult to throw that into the 2.0 spec. There would be a “fixed-function” equivalent that you can still rely on, and implementations need not support it in hardware. But, when the time comes that these things are avaliable in hardware, it would be better to have the functionality already in place than to have to rely on extensions again.

I’m not saying that programmable texture filtering is something that has no place in OGL. What I am saying is that you can accomplish the exact same functionality with the currently existing vertex and fragment shaders. In other words, texture filtering operations can be and should be done in the fragment shaders. And that to me seems to be a good reason to not add extra weight to the OpenGL 2.0 specification, which is trying to simplify things, not complicate them.

If there is no way to make fragment shaders work on current or near-term hardware, then there is no point in making OpenGL 2.0; it’s a waste of everybody’s time, since no one will be able to implement fragment shaders.

The point of this is that 2.0 is supposed to be future looking. It is reasonable to believe that programmable texture filtering (as separate from fragment programs) is something that may happen in 3-5 years (if not sooner).

It seems to me like you are saying the OGL 2.0 should be both compatible with current video cards and support features that only future cards will have a chance of having.

Have you read the OGL 2.0 whitepapers yet? They practically are a specification already, and I’ve taken most of my arguments from what I have understood about OGL 2.0 from reading those papers.

j

I’m not saying that programmable texture filtering is something that has no place in OGL. What I am saying is that you can accomplish the exact same functionality with the currently existing vertex and fragment shaders. In other words, texture filtering operations can be and should be done in the fragment shaders. And that to me seems to be a good reason to not add extra weight to the OpenGL 2.0 specification, which is trying to simplify things, not complicate them.

That they can be done in fragment shaders is not (quite) in dispute here. Whether or not they ought to go there is the matter at hand. Why would it matter if the API has a few calls that are not implemented on most, if not all, hardware? It then becomes immediately obvious when you want a texture filtering effect vs. the sort of things that go on in a fragment shader.

It seems to me like you are saying the OGL 2.0 should be both compatible with current video cards and support features that only future cards will have a chance of having.

Kinda. Having programmable texture filtering doesn’t force anyone to use it. It only means that, if you do use it, the implementation will default to software rendering. Maybe no OpenGL 2.0 card ever implements them in hardware, but they are avaliable in software if needed.

However, forcing all implementations that use fragment shaders to allow for virtually infinite texture accesses is ludicrous. That means that, by the time OpenGL 2.0’s specification is finished, no card will be able to implement it. For academic purposes, 2.0 would be nice, but for the purposes of creating a product that requires high-performance graphics, it is utterly worthless. By contrast, when D3D 9 launches, certainly nVidia, if not ATi, will have a card that supports the 9.0 pixel shaders.

As a software developer, which would you choose? An OpenGL 2.0 API that is utterly worthless as far as letting you get to the avaliable hardware, or a D3D 9 API that is actually useful.

OpenGL 2.0 should be scalable to the future, but not at the expense of the present. Forcing an implementation of fragment shaders to be able to infinitely access its textures is very different from allowing fragment shaders to do that.

However, forcing all implementations that use fragment shaders to allow for virtually infinite texture accesses is ludicrous.

Ludicrous? I prefer ‘forward looking’.

That means that, by the time OpenGL 2.0’s specification is finished, no card will be able to implement it.

Yes, and? You seem to forget that it’s only recently that consumer cards have implemented the whole of OpenGL1.x.

For academic purposes, 2.0 would be nice, but for the purposes of creating a product that requires high-performance graphics, it is utterly worthless.

That doesn’t follow. I would know current cards don’t support single texture multiple access, so I don’t use it. Problem solved.

That doesn’t follow. I would know current cards don’t support single texture multiple access, so I don’t use it. Problem solved.

And therein lies the problem. That is not true. The Radeon 8500 can access a texture twice. Basically, what will happen is that every card will support a different number which cannot be retrieved programmatically (because 2.0 assumes that multiply accessing a texture is commonplace, so it will not have some kind of glGetMultipleAccess). Therefore, you will either have to ask for the name of the implementation and infer how many times you can do it based on information from the developer, or you will simply assume the least-common denominator. Neither of these are reasonable responses to the problem at hand.

There is a third option, which is to assume that if the hardware you are working on supports OGL 2.0, then it will be able to handle as many texture accesses as you feel like asking it to. I think that OGL 2.0 compliant graphics cards are going to be quite different from current accellerators, with somewhat different rules. They will act more like massively pipelined CPUs instead of the GPUs that we know today. Everything will be more general purpose, so instead of worrying about how to tweak the pipeline to achieve results with limited resources, programmers will simply be able to program what they want pretty much the same way they program for any Intel or AMD processor.

Considering that the fragment shader language in OGL 2.0 has for loops, function calls, and branching if statements, I don’t think that a card which implements it will be limited by the number of times it can grab some data from its memory. Especially since a card must be able to handle all of the above to be considered compliant, not just a subset.

j

Some general comments, rather than try to answers specific points raised in earlier posts:

The common filtering algorithms are easily expressed in hardware and lend themselves to economical fixed function implementation. The only realistic options here are to allow bicubic type filtering where the user gets to specify the weights.

To make this operation programmable will involve adding Fragment Shader like features (and size) to the filtering hardware, that for many people will remain under utilized.

The general nature of the Fragment Shader allows the user to implement whatever filtering algorithm they want to and built-in functions will return the LOD and gradients to save having to determine these yourself. This will never be as fast as dedicated hardware so programming a filter supported directly by hardware will always be slower, but if you have some fancy 30 tap filter you must have then it is clearly the way to go.

OGL2 does not limit you to a single sample to a texture map and while some hardware does impose limitations these will be going away, if only because DX is also removing this limitation.

When we were formulating our ideas for OGL2 we had to balance up looking forward with what could be achieved reasonably soon. I don’t think you will have too long to wait for some of these features to exist.

Dave Baldwin
3Dlabs