mutable texture formats

In the newer d3ds (10 and up) there is a mechanism whereby one can effectively change an texture format - it is called “views” there.
This is hardly one of the more useful features. Actually I only know of one practical use, that is, to change from srgb to non-srgb and the other way around (which was just a sampler state in previous directx-es).

Anyway, as the various vendors actually support the newer d3ds, and with them this particular feature (the mutable texture formats), why can’t we access this hardware capability from opengl?
So i suggest that we make the texture formats “mutable”. That is, once the texture is created, its format can be changed with glTexParameteri.
The sampelr objects will also possess a format state, which will override the texture’s, just like the other parameters.
Of course we will have to divide the various formats into groups of mutual compatibility. Then a texture’s format can be changed only to one of those that are from the compatibility group of the original format.
If a sampler’s format is not compatible with the bound texture we act as in other similar cases - e.g. generate INVALID_OPERATION on the subsequent draw commands.

The format is actually property of the texture levels and not of the texture object itself, but this is so only for historical reasons. (different mip formats within the same texture is possible but it only makes the texture unusable and is a burden for the drivers to deal with).
Changing the format with glTexParameteri will apply to all mips at once.

This will give the full functionality of the directx-es, but with much simpler and intuitive api (the directx way is way too complex and bloated with the numerous pointless objects you have to create and manage).

What do you think?

The biggest flaw with the D3D10/11 approach is that creating a view is mandatory - there is no default view that you can just grab and use. It’s a one-time-only op at creation time for sure, and anyone sensible will end up wrapping it with something more appropriate to their program’s usage requirements, but mildly annoying all the same.

On the other hand, it does allow for unification of a whole bunch of previously different resource types, and pretty much everything is now some variation on buffer or texture (even the depth buffer is just a regular Texture2D). It’s actually quite an elegant setup really - a resource is the raw object and a view defines how the pipeline interprets that object, so there’s a nice and clear separation going on. The initial (which are the current) implementations are a mite clunky, but like everything in D3D land they can be expected to get better over time. So for standard texturing you have a ID3D11Texture2D with an ID3D11ShaderResourceView created on it, but you can also create an ID3D11ShaderResourceView on an ID3D11Buffer object, allowing for easy render-to-vertex-buffer or texture-from-vertex-buffer if that’s what you wanted to do.

In many ways it behaves a lot like OpenGL’s buffer object binding points, with a little bit of glTexParameter thrown in, in other words. So there’s likely no requirement for OpenGL to specify any kind of full implementation of D3D views, with the specific use case you identified (mutable texture formats) being the only obvious one, and correctly expressed as a glTexParameter or glSamplerParameter. So long as the mandatory “you must create a view” requirement didn’t exist, it sounds fine and reasonable.

As it is now, the texture format is not so well-defined because it is specified separately for each mip. The texture is considered complete if all mips have the same format, but after that the user still can change some mitp’s format and texture again becomes incomplete.
Then it may be difficult to define which is the “original” format to which we want any new one set with glTexParameteri to be compatible.
For this reason we may want to allow “mutable formats” only for the “immutable textures” created through the new extension GL_ARB_texture_storage.

Exactly how would this work? What would it mean to change the format of a texture from GL_RGBA8 to GL_RGBA16? I don’t know much about how D3D10+ works in these cases.

My concern is that there won’t be very many valid re-interpretations of data, and even fewer useful ones. For example, you could turn GL_RGBA8 into GL_RGB10_A2 or GL_R32F or something. But what exactly does that gain you in terms of real usefulness? What problem does this solve?

Also, there’s the issue of specifying behavior, essentially forcing implementations to do things the D3D way. I don’t care much for that idea.

Most importantly, you said, “I only know of one practical use, that is, to change from srgb to non-srgb and the other way around (which was just a sampler state in previous directx-es).” We already have that, though only as an extension (EXT_texture_srgb_decode) for the time being. This seems like a far less intrusive way of getting the useful functionality of this concept.

OpenGL doesn’t have to expose every possible thing that hardware could do. It just needs to expose all of the useful things it can do.

Reviewing the DXSDK, 10/10/10/2 formats aren’t in the same type family as 8/8/8/8 so that’s either one of the nice arbitrary restrictions that D3D likes to hit you with every now and then, or there was a practical reason behind it. The only thing that would have given you is a psychedelic screen effect without any post-processing, which is probably not in very high demand.

In reality the purpose of views in D3D is something entirely different, and mutability of texture formats is a side effect rather than the main objective (which was separation of resource definition from how the resource is to be used, and which allowed for more generalization of resource types - think of it as being kind of like mallocing a void * buffer then casting it to a struct type). It’s also a limited mutability rather than full general, so you can’t covert the example of GL_RGBA8 to GL_RGBA16 - they must be from the same type family, which tends to mean same number of components and same component size.

If GL were to get this a more general mutability would have one practical use I can think of, and that’s where you might have a texture that sometimes you want to access as RGBA8 and sometimes as R32F. Say you want to interpret one portion of it as depth info and one portion as colour info, and - because the texture is quite large - resource constraints prohibit you from creating two of them. True, it’s a mite far-fetched, and true, you could do some fragment shader packing/unpacking (at the cost of a few extra instructions), but it is sneaking into the realm of things that could happen.

If GL were to get this a more general mutability would have one practical use I can think of, and that’s where you might have a texture that sometimes you want to access as RGBA8 and sometimes as R32F. Say you want to interpret one portion of it as depth info and one portion as colour info, and - because the texture is quite large - resource constraints prohibit you from creating two of them. True, it’s a mite far-fetched, and true, you could do some fragment shader packing/unpacking (at the cost of a few extra instructions), but it is sneaking into the realm of things that could happen.

It seems to me that the way to handle this would be with a more flexible type system, something that would allow you to build an image format from raw components. You could create an image format that would be the equivalent of GL_R16F_GB8. I don’t know how you would handle shadow accesses from such a texture, as those explicitly return a single float value.

Even so, it’s not of that much utility. If you need that level of flexibility, nobody’s stopping you from using special shader logic to turn one texture’s format into another. You could unpack the RG components of an RGBA8 texture into a float to emulate GL_R16F_GB8.

I don’t know what was the purpose of the views that d3d designers had in mind (im not even sure if they had anything in mind at all), but the mutability of the texture formats is the only real technical effect of all this complex api.

If opengl was to gain this feature, then i guess GL_RGBA8 and GL_RGBA16 would not be in the same compatibility group (they are not in d3d).
But GL_RGBA8, GL_RGBA8I, GL_RGBA8UI, GL_RGBA8_SNORM and GL_SRGB8_ALPHA8 would all be in the same group.

As for EXT_texture_srgb_decode, it would be great if ATI supported it.
Mutable formats would be more general than the EXT_texture_srgb_decode.
I don’t know how much more useful they would be, but if they are available the people probably will think of new useful tricks that can be done with them.
My particular interest in them is related to portability from d3d to opengl.

The purpose of views in D3D is to enable generic backing storage to be interpreted in different ways, nothing more and nothing less. Like I said, it’s a cast. Take this chunk of storage and use it as a texture, take that chunk and use it as a depth buffer, take the other chunk and use it as a render target, take a chunk that was previously used as a depth buffer and now use it as a texture (or vice-versa), but the specification and allocation of the backing storage remains reasonably agnostic to how it’s going to be interpreted. Whether it was the correct way (or even a good way) to go about implementing this capability is neither here nor there for the purposes of this discussion.

Limited format mutability is purely a side-effect of this. I seriously doubt if the designers had it in mind as an explicit goal; it seems more of a “hey, you can do this too” kind of thing.

The specific example I gave above would be part of a more general capability, which would include being able to use different texture formats in the same texture atlas. So one subrect of the atlas could be RGBA8, another subrect could be RGB10A2, a third subrect could be R32F, and so on. That on it’s own would extend the capabilities of texture atlases in a useful and interesting manner, and would mean that in order to interpret each subrect correctly you could do so without needing to change shaders. The principle could in theory be extended to texture arrays as well, which would have the effect of lifting one restriction associated with arrays, and again without the need to change shaders.

Should OpenGL have any business defining how anything as specific as a texture atlas may be used? That’s open for discussion. What is certain is that one person’s resistance to the idea (and bearing in mind that this is the same one person who was resistant to the idea of separate shaders) does not make it a bad idea. Nor does it make it a good one, but it does remain one that is worth some further discussion.

def OffTopic:

Who's that?

What you say does not appear to be the case. The create functions are CreateBuffer, CreateTexture2D, CreateTexture3D, etc. There is no CreateGenericStorage or something like that.
Next, i don’t see what “casting” the views are needed for. I mean, whats the difference between binding a texture view to a sampler instead of just binding a texture to the sampler?
The ONLY thing the view is adding is it’s ability to override the texture format. There is absolutely nothing else. But (as per my suggestion) the format changing could be done without the views.

If you mean you can bind the same object (texture) for different things - once as a texture, then as a color buffer, then as something else, this already can be done in d3d9 and in opengl
even though there are no views there. (in the newer opengl there is yet another set of binding points - image units). So we don’t need views for this either.

The views don’t appear to be a meaningful additions to the api. A pure bloat.
I think they are added for political rather than for technical reasons. Something like “keep adding more COM objects guys. the more they are, the less portable our api, the better for us. every bit counts”.

I personally think you’re being a mite paranoid, not to mention showing some prejudice, over this. I wouldn’t try to defend COM as an architecture - it’s ghastly - but COM itself has nothing in it that prevents portability. It just uses standard C/C++ constructs, and as such can be implemented on any platform. This kind of approach only serves to weaken your argument. That aside, you seem to be missing the documentation for D3D11_RTV_DIMENSION, D3D11_SRV_DIMENSION, etc, which is the crucial part that implements the cast.

in d3d you create render target view by specifying texture and region in this texture, i.e. face for a cube texture, a slice for a 3d texture and array texture, etc. then bind the view as render target.
in opengl you do all this at once - the command with which you bind the color/depth/stencil buffer also takes parameters that specify which face, which slice, etc.
how is the opengl’s direct way worse than the views?
in d3d you have additional objects to work with, which is more code, more opportunities for bugs (like leaks if you forget to release them), etc. for no benefit over opengl. don’t you agree?

I’m not commenting on whether the implementation is good or bad, just on what appears to have been the thought processes behind it. Personally I can see the utility of views in D3D, from the perspective of D3D’s own capabilities, but feel that the current implementation of them is a lot clunkier than it should have been. But that clunkiness does not take away from the utility, and nor does the utility of a D3D-specific implementation of something mean that GL should implement the same thing in the same way (or vice-versa, of course).

I was not arguing that their views are a bad implementation, but that they are useless in principle :stuck_out_tongue:
Anyway there is really no point to argue over this, doesn’t matter.
I only hope the GL design guys wont take example from microsoft how to do it, should they wish to add such feature in the GL.

It seems they actually added this functionality in 4.3. They did it in somewhat more complex way than what I was hoping, but it’s still better than microsoft. At least we don’t have new kind of objects.

I would rather have them called “texture aliases” than “texture views”. that would be more correct as thats what they actually are. also would be less copycating-of-dumb-microsoft-terms. Oh well.

Well, there not really aliases. Yes, they can be called so if you only change the interpreted internal format, however you can also select a subrange of mip levels and layers thus you actually select a subset of the texture images inside the texture. This way the “view” term is more appropriate.

A “view of a texture” would imply that it is NOT a texture itself but something different that references a texture (as in d3d).

But the new objects we have are really textures on their own right - after you create one, you use it exactly as you would a normal texture.

They are just textures whose storage memory aliases that of another texture.

As for that you can specify mipmap subrange - so what?
Where in the definition of the programming term “alias” it is stated that it should exactly coincide with the other object?
They can only partially overlap

Moreover the mip range for the “view” makes much less sense - why a “view” should have it’s own mip range if it is just a “view” to another texture?
When someone “looks” at a texture, he sees a texture with 5 mips. But when someone else “looks” at the same texture at the same time, he sees 3 mips!

Maybe the better way to look at is that the new texture call says this texture uses the exact same backing store of memory as an existing texture. Since that is what it is doing. By being a texture in it’s own right, it makes it heck-a-simple to specify all the texture state, swizzles varying, mip map ranges, lod biases, etc… blah, blah blah. I think that which was added was great… to me it makes crystal clear sense: “new texture’s texel data references same memory for the texel data as an existing texture”.

Moreover the mip range for the “view” makes much less sense - why a “view” should have it’s own mip range if it is just a “view” to another texture?

Actually, it makes a lot of sense. There is the object (the texture’s storage), and there are views of it. A view of an object can theoretically see less of the object than actually exists, if something is in the way. It sees some of the object, but not all of it.

Thus, a texture view can see less of the texture than the original texture does.