PDA

View Full Version : OT: HL2 FSAA prob, what did they mean?



GPSnoopy
07-19-2003, 02:50 PM
I'm sure you've read this news on other website. It's a bit OT I know, plus it's a DX engine.
But I was really wondering about what they said:


It's a problem for any app that packs small textures into larger textures. (Valve)

Did they mean that they use like one big texture that actually holds multiple textures?
Now why would you want to do that?

MikeC
07-19-2003, 03:20 PM
Now why would you want to do that?

To reduce the number of render state switches.

It's not just the cost of texture binding - being able to render more stuff without changing renderstate means fewer, larger batches of rendering primitives, and this is usually touted as the key to good performance these days.

Ostsol
07-19-2003, 07:09 PM
And in case you want to know. . . here's a bit by ATI's sireric on the problem and on why it's likely that only ATI will be able to have a solution:

It's not a special mode of AA. It has to do with the method used to sample textures. If you render a partial pixel, you need to sample the texture at that pixel to get a color for the fragment. What texture coordinate do you use for a partially covered pixel? You can use the pixel center (which is what is done in current drivers/specs), or, the R3xx allows you to sample at the centroid of the fragment.

Both systems have problems. Center of pixel can lead to sampling outside the texture, and then the application has to create a buffer region outside to compensate (but that buffer region can be unlimited, depending on the pixel/texel ratio). If you use centroid, you avoid that problem, but can create new ones in cases where you only want to sample a texel once (i.e. menus and text). We would need to way to allow the application to specify the kind of sampling wanted on each texture. That's not in the API now. However, the HW can do it. We might be able to do something in the control panel.

zeckensack
07-19-2003, 07:16 PM
Brilliant.

Epic doesn't understand texture filtering.

Valve doesn't understand texture filtering.

Who's next?

From a signal theory pov (which is the only way to go for texture filtering), "centroid sampling" or whatever they like to call it is pure nonsense.

Q: My audio samples exhibit an annoying click at the end. What can I do?
A: Duplicate and repeat the last sample.

*shudder*

Korval
07-20-2003, 12:38 AM
I'm agreeing with Zeckensack here, unless the above post was made by people not directly involved with the game in question. What they're saying really doesn't make sense. And it doesn't make sense on several levels.

Adrian
07-20-2003, 01:50 AM
What I find really bizarre is that they only just noticed this was a problem? The game's been in development for several years and 70 days from release they announce they are trying to resolve an issue with FSAA? If they didn't just notice it why couldn't they have fixed it for the NV30?

I mean I never use FSAA except I test my software with it and with AF and multisampling. The reason being that those settings have a habit of causing me problems.

Jan
07-20-2003, 01:55 AM
I donīt know much about AA, but i always thought itīs done on the framebuffer and has nothing to do with texturing, at all, does it?
And i donīt understand what AA has to do with texture-filtering. Could someone explain this to me?

Jan.

Korval
07-20-2003, 02:05 AM
I donīt know much about AA, but i always thought itīs done on the framebuffer and has nothing to do with texturing, at all, does it?

Well, there are antialiasing techniques that aren't all done in the framebuffer, but they aren't discussing those techniques.


And i donīt understand what AA has to do with texture-filtering. Could someone explain this to me?

Which is why we're on the belief that they're idiots. There really isn't a connection there, but they believe there is one.

GPSnoopy
07-20-2003, 03:02 AM
Originally posted by MikeC:
Now why would you want to do that?

To reduce the number of render state switches.

It's not just the cost of texture binding - being able to render more stuff without changing renderstate means fewer, larger batches of rendering primitives, and this is usually touted as the key to good performance these days.


I don't think that nowadays it's the real bottleneck, especially when using tons of shaders. I mean there are a lot of posts around here where NVIDIA guys have been telling that a front to back rendering was more important than reducing the number of state switches.
Packing small textures into larger sounds to me like it's bringing more problem that it solves: limits filtering techniques, mipmaps troubles, and by extension this FSAA problem.

I think it's a bad idea not to at least offer the choice to switch off this "texture packing".

What really bothers me in this situtation is that Valve is now blaming NVIDIA, ATI and Microsoft/DirectX for the consequences of their decisions.

zeckensack
07-20-2003, 03:19 AM
Originally posted by Jan2000:
And i donīt understand what AA has to do with texture-filtering. Could someone explain this to me?

Jan.[/B]If your renderer breaks upon changing resolution, it will break upon activating AA. This is a very general statement, different things can go bad depending on the exact AA method.
But it's about the only thing that's true about potential AA issues.

(well, there are obvious issues with R2T postfilter effects, but let's not complicate things further)

Adrian
07-20-2003, 03:21 AM
NVIDIA guys have been telling that a front to back rendering was more important than reducing the number of state switches.

They're both important, impossible to say which is more important because it depends whether your app is fill rate or CPU limited. If your app is fill rate limited then front to back rendering will help. If your limited by CPU or the bottlenecks introduced by state switches then clearly those need to be reduced.

Personally one of the most obvious optimisations with regards to texturing is to reduce bindtexture calls. It would be near the top of my optimisation list to pack textures into fewer big textures. I don't think it was a poor decision by valve to do that, I would have thought most commercial games do the same.

evanGLizr
07-20-2003, 09:46 AM
Originally posted by Jan2000:
And i donīt understand what AA has to do with texture-filtering. Could someone explain this to me?

Jan.

It has to do at polygon edges: if you multisample (i.e. you do only edge AA, as opposed to supersampling), you may endup sampling outside the polygon, which may cause sampling outside the texture if it's "tightly" snapped to the poly.

Thus, if you are packing different smaller textures inside a bigger texture, with this kind of AA you may endup sampling texels from a neighbouring smaller texture.

MZ
07-20-2003, 04:35 PM
I must admit so far I almost ignored parts of specs regarding multisampling, so excuse me if I say BS.

It has to do at polygon edges: if you multisample (i.e. you do only edge AA, as opposed to supersampling), you may endup sampling outside the polygon, which may cause sampling outside the texture if it's "tightly" snapped to the poly.

Thus, if you are packing different smaller textures inside a bigger texture, with this kind of AA you may endup sampling texels from a neighbouring smaller texture.Are you sure about this? Let's forget about packing small textures for a while. Take simple black-white, 2x2 chessboard texture, with wrap mode set to "repeat" (so that image is tilable, but discontinuous at edges). Let's render textured quad, having texcoords covering entire texture {(0,0), (1,0), (1,1), (0,1)}. Now, If what you said was true, such quad rendered in MSAA mode could have awful errors at edges? Can't believe multisampling is _that_ broken... If HW is supposed to "reject" samples failing depth and stencil tests, then why would it ever be allowed to "accept" samples falling outside of polygon?

Going back to HL2, could somebody explain this:

1) Would simple 1-texel-wide border of each packed sub-texture solve the problem? (the border filled with data according to desired wrap mode of sub-texture)

2) Is this problem specific to DX? DX rules of bilinear sampling ar said to be different in details from these of GL.

Ostsol
07-20-2003, 06:14 PM
Here's some screens of a little test-app:

No FSAA: http://members.shaw.ca/dwkjo/screenshots/check.jpg
4x FSAA: http://members.shaw.ca/dwkjo/screenshots/check2.jpg

Filtering: GL_NEAREST
Texture wrap: GL_REPEAT
Texture size 64x64

Radeon 9700 Pro, Catalyst 3.6

As I was about to post this I tried a 256x256 version of the texture with GL_LINEAR filtering. The result was that the bleeding occured regardless of FSAA unless the quad was not rotated at all.

[This message has been edited by Ostsol (edited 07-20-2003).]

*Aaron*
07-20-2003, 06:57 PM
Thus, if you are packing different smaller textures inside a bigger texture, with this kind of AA you may endup sampling texels from a neighbouring smaller texture.Why use edge AA when you can't use mip-mapping? This texture packing technique would cause that problem even if you had a border around the subtextures. AA, which can be an expensive technique, only gives a small image quality improvement. Mip-mapping, which is not that expensive (and can even lead to speed increases when the texture stride is large), gives a noticable image quality improvement.

Ostsol, that example is designed to show the bleeding at the edges; it's not a realistic situation. Most tiling textures don't have such abrupt color transitions that would show at the edges as the checkerboard texture does. And as you pointed out, with this texture, this artifact occurs even with GL_LINEAR magnification filtering, which has been standard for years now.

I think the texture packing technique is a fundamentally flawed technique because of the mip-mapping issue, and I don't think it is/will be widely used. Really, why should it be such a performance hit to bind a new texture?

Madoc
07-20-2003, 10:38 PM
It's not about texture binds, it's about packing state changes so you end up having shaders/materials that cover larger batches of polygons. Similalry you might use vertex components or textures to change material properties rather than changing state outside drawprimitive calls.
Rendering fewer batches is supposed to be saving you a lot of CPU time.
AFAIK this is a more serious issue with D3D than OGL.

davepermen
07-21-2003, 01:33 AM
Originally posted by *Aaron*:
I think the texture packing technique is a fundamentally flawed technique because of the mip-mapping issue, and I don't think it is/will be widely used. Really, why should it be such a performance hit to bind a new texture?

if you have to switch nearly per triangle, it would hurt because you cannot render batches.. thinking of old-style lightmapping now.. i dunno why/how it hurts hl2 exactly, though..

Nutty
07-21-2003, 02:19 AM
I don't think that nowadays it's the real bottleneck, especially when using tons of shaders. I mean there are a lot of posts around here where NVIDIA guys have been telling that a front to back rendering was more important than reducing the number of state switches.
Packing small textures into larger sounds to me like it's bringing more problem that it solves: limits filtering techniques, mipmaps troubles, and by extension this FSAA problem.


If you have say a couple thousand non-power of 2 very small textures, then it makes very good sense to pack them into large power of 2 textures. Provided you group textures together as to their render method.

To me it sounds like they aren't leaving pixel gaps between their images in packed textures. Anyone who's written a texture packer will obviously know you have to do this, just to stop bilinear filtering sampling adjacent packed images.

Perhaps they might need to increase the texel gap, depending on the amount of sample coverage used in the FSAA implementation.

But I agree, its a bit pointless waiting until this time, then blaiming the IHV's when your engine design breaks under certain conditions. http://www.opengl.org/discussion_boards/ubb/rolleyes.gif

[This message has been edited by Nutty (edited 07-21-2003).]

zeckensack
07-21-2003, 05:40 AM
Originally posted by Ostsol:
As I was about to post this I tried a 256x256 version of the texture with GL_LINEAR filtering. The result was that the bleeding occured regardless of FSAA unless the quad was not rotated at all.Technically it shouldn't. This is most likely a precision limitation inherent to the tex coord interpolators. That's a different issue.

However, I'd like you to check a few things in your test app:
1)Have you tried with a mipmapped checkerboard texture?
2)What's the texel/pixel ratio, roughly? Do you magnify or minify?
You see, GL_LINEAR is not a particularly useful minification filter http://www.opengl.org/discussion_boards/ubb/wink.gif

Minification will lead to texture aliasing (shimmering) either way with GL_LINEAR, regardless of AA.

evanGLizr
07-21-2003, 05:43 AM
Originally posted by *Aaron*:
Why use edge AA when you can't use mip-mapping? This texture packing technique would cause that problem even if you had a border around the subtextures. AA, which can be an expensive technique, only gives a small image quality improvement. Mip-mapping, which is not that expensive (and can even lead to speed increases when the texture stride is large), gives a noticable image quality improvement.

First, normally you don't mipmap the textures you pack. Look at Quake3, for example, it packs lightmaps but because a lightmap is almost always rendered magnified, it doesn't bother mipmapping them. The non-mipmapped texture lookup is also faster to do by the graphics chip, so from a performance point of view (in this case!) it's also better not to use mipmaps.

Second, even if you mipmap those textures, when the texture is used minified you are less likely to be able to notice that problem because it will affect to few pixels and without a visible pattern. One of the reasons is that the textures packed are never tiled (or at least you cannot use repeat modes, although the artist could tile them "manually").

Regarding the binding cost:
- in OpenGL there shouldn't be much of a difference, obviously the least unnecessary work you do, the better and if your native textures are not power of two, you will get a better texture cache usage if you pack them in power or two textures.
- in DX, on the other hand, it's very important to be able to send large batches to the graphics board, because each drawprimitive call means a ring 3 - ring 0 context switch. Packing textures allows you to send larger batches (as already mentioned by other posters).

Won
07-21-2003, 07:56 AM
zeck --

I'm curious what you mean by the sampling theory stuff on top. My understanding is that the issue is not an aliasing problem, but an offset problem (like jitter or phase error). In multisampling, the geometry and texture sample locations are in different locations, and the discrepancy projected onto texture space can be arbitrarily large (depending on the screen slope of the primitve). So texel boundaries in your packed textures will only have limited success there. Is this true?

I am also curious about the rotated quad screenshots above, particularly the 256x256 phenom. It's my understanding that the minimum texture coord precision is 10 bits, so 256x256 shouldn't be a problem. Maybe it's because texels are smaller, so in texture space the discrepancy is magnified?

I don't know much about packing textures, but I'm wondering if you can clamp the LOD to enable mip mapping on packed textures, so long as the packed textures are on some power of 2 boundary. I guess this would probably wreck havoc on texture coordinates etc.

Miscellaneous facts about MIP-mapping:

You can use mip-mapping to guarantee no aliasing, but not using mip-mapping will not necessary guarantee aliasing. If you sample a low-frequency texture (such as a lightmap), as long as you don't drop below that sampling frequency (which is capped by texture resolution, but otherwise independent) you will not get aliasing.

Usually, for textures that are going to be used in a variety of different resolutions, mip-mapped textures perform considerably better due to heightened sampling coherence. This doesn't have anything to do with "texture stride" which doesn't make sense except in the case for rectangle textures, which can't be mip-mapped. In video memory, textures are tiled/swizzled. This is as simple as performing permutations on the address wires so it is at little cost to the hardware. It's possible to treat frame buffers this way as well.

-Won

[This message has been edited by Won (edited 07-21-2003).]

Xmas
07-21-2003, 08:36 AM
IMO there's one important use for packed textures. And that's using them for keeping a tileset. Like for tiles on terrain (and font rendering of course).

Ostsol
07-21-2003, 08:58 AM
Originally posted by zeckensack:
Technically it shouldn't. This is most likely a precision limitation inherent to the tex coord interpolators. That's a different issue.

However, I'd like you to check a few things in your test app:
1)Have you tried with a mipmapped checkerboard texture?
2)What's the texel/pixel ratio, roughly? Do you magnify or minify?
You see, GL_LINEAR is not a particularly useful minification filter http://www.opengl.org/discussion_boards/ubb/wink.gif

Minification will lead to texture aliasing (shimmering) either way with GL_LINEAR, regardless of AA.
Okay, I tried it again. . .

256x256 texture, looks like the one in the screenshots I posted.

glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_MIPMAP_NEAREST);
glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR_MIPMAP_LINEAR);
gluBuild2DMipmaps(GL_TEXTURE_2D, 3, pBitmap->sizeX, pBitmap->sizeY, GL_RGB, GL_UNSIGNED_BYTE, pBitmap->data);

I set the viewport to 512x512 pixels and set glOrtho's parameters to twice the size of the quad, so as to render the texture with a 1:1 pixel:texel ratio.

The result was that the bleeding still occured regardless of FSAA.

[This message has been edited by Ostsol (edited 07-21-2003).]

*Aaron*
07-21-2003, 10:19 AM
Usually, for textures that are going to be used in a variety of different resolutions, mip-mapped textures perform considerably better due to heightened sampling coherence. This doesn't have anything to do with "texture stride" which doesn't make sense except in the case for rectangle textures, which can't be mip-mapped. I got this from an nVidia document on improving performance. By "texture stride", I was referring to the texel/pixel slope. I'm not sure where I came up with the term "texture stride", but I can't seem to find it with a google search, so I probably just made it up. The reason given in the document was that mip-mapping can increase cache coherency. For example, if a graphics chip stores 8 texels per cache line, and the slope is 8 texels/pixel, you have to read in a new line for every pixel. Mip-mapping minimizes this slope. Perhaps we're both talking about the same thing.

I found an nVidia document about maximizing texture performance in D3D that talks about the issue of storing lightmaps as subtextures and the bleeding issues that arise. It doesn't say anything about FSAA, though.
http://developer.nvidia.com/docs/IO/1409/ATT/GDC_Texture.ppt

V-man
07-21-2003, 10:30 AM
----glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR_MIPMAP_LINEAR);----

hmmmm?

Won
07-21-2003, 10:32 AM
Yeah, that's an odd MAG filter, considering mip-mapping doesn't do anything int his case.

What happens when you offset the quad by fractional pixel amounts?

-Won

zeckensack
07-21-2003, 10:35 AM
Originally posted by Won:
[B]zeck --

I'm curious what you mean by the sampling theory stuff on top. My understanding is that the issue is not an aliasing problem, but an offset problem (like jitter or phase error). In multisampling, the geometry and texture sample locations are in different locations, and the discrepancy projected onto texture space can be arbitrarily large (depending on the screen slope of the primitve). So texel boundaries in your packed textures will only have limited success there. Is this true?That's essentially the issue. A (multisample) subpixel ends up covering a complete pixel after downfiltering. Relative position information inside this pixel is lost. So an assumed single subpixel ends up on the pixel center, this is also the position for which the texture sample is generated. The problem is, that the actual subpixel position before downfiltering is inside the polygon edge, but its position after downsampling (and accordingly the position used for texture lookup) is outside the polygon.

Re the signal theory stuff:
If your transfer this problem into a frequency domain, the current multisampling approach is the right thing. Your screen pixels form an ordered grid, ie a constant and known maximum representable frequency (think 'sampling rate').
Mipmapped texture sampling is a means to use this frequency headroom to its fullest, without exceeding it (That would lead to texture aliasing aka shimmering; this is exactly the same reason why an audio A/D-converter must have a low pass filter at the analog input).

For this to work, the source material should fully use its own frequency headroom, ie the smallest texture features should be one sample in size. You should not apply a 'smooth' filter to textures, because that reduces frequency. This is why I made the Epic comment, they do exactly that and then 'compensate' the resulting bluriness with a negative lod bias. This is just plain stupid.

Back to the issue at hand, "centroid" sampling punctually reduces signal frequency at the offending edges by stretching out the source material. This deforms the sampling grid, and creates a 'black hole' in the frequency domain. All the edge samples will be comparatively low frequency (low contrast to their poly interior neighbours) ... even though the frequency of the screen grid is not warped in any way. This cannot be right.

This most obviously compares to older movies' bluescreen edge artifacts, when the effect is done at too low a resolution. This should cover the screen space edges.

The more interesting problem (to me, anyway) appears where two 'centroid' sampled edges belonging to the same object connect. A proper model would have its tex coords laid out so that the sampling would straddle this edge in a linear fashion. The transfer function 'texture frequency'->'screen frequency' is not affected by the number of interior edges.

"Centroid" creates breaches in the transfer function, where part of the source material is underrepresented in the sampled result. This is also a form of aliasing.
This issue is also already known, it manifests when you render a large 2D image w several independant tile textures, where you need to take some precautions for the result to look right (texture borders). A simple GL_CLAMP_TO_EDGE will not work. Well, centroid is a glorified form of clamp to edge, so I expect the same thing to happen.

Elixer
07-21-2003, 10:49 AM
Hmm, I also wonder if all those teasers they released before that statement showed any of these problems?

I assume they just started testing FSAA and said "OOPS!, we can't fix this in time, so lets start the blame game!" ?

Maybe they should switch back to openGL http://www.opengl.org/discussion_boards/ubb/wink.gif

M/\dm/\n
07-21-2003, 11:01 AM
HMMMM, NV hw has similiar problems in Splinter Cell, though I remember that with oldest drivers, when FSAA wasn't disabled, there was no problems.

BTW, how this texture packing releats to 8x1 4x2 pipe design?

Ostsol
07-21-2003, 11:21 AM
Originally posted by V-man:
----glTexParameteri (GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR_MIPMAP_LINEAR);----

hmmmm?

Originally posted by Won:
Yeah, that's an odd MAG filter, considering mip-mapping doesn't do anything int his case.

What happens when you offset the quad by fractional pixel amounts?

-Won
Yeah. . . I messed up with copy-pasting and reusing old code that I must have been experimenting with and managed to screw up. http://www.opengl.org/discussion_boards/ubb/smile.gif

Anyway, I translated the quad by 0.1 along the x-axis (with no rotation and no FSAA) and there was some bleeding, again.

Won
07-21-2003, 11:40 AM
zeck --

OK. Let's see if I first have an understanding of what is meant by "centroid sampling." In the ATI-guy's description does "centroid of the fragment" mean the centroid of the portion of the primitive that is contained within the pixel? How does this become a problem for menu/text? And your particular complaint is because in centroid sampling the texture sampling mesh is offset, disturbing the regular sampling?

...

About the plain stupidity of Epic, there may actually be a semi-intelligent reason. This is because a bilinear filter doesn't make a very good low-pass filter, and performing the smoothing off-line essentially let you control the filtering better. From a performance standpoint, it is probably not a good idea though considering you are essentially intentionally screwing with texture caching/prefetching and eating up video memory. Stupid? Depends. Tim Sweeney is a pretty smart guy.

Also just to be more precise, ADCs don't need to have low-pass filters; they can work with bandpass filters as well as long as you don't exceed the total bandwidth (not the nyquist frequency) of the ADC.

-Won

FXO
07-22-2003, 01:41 AM
If we put aside the packaging of textures to a big one and look at the screenshots Ostsol posted, the problem is more obvious.

This is not a problem for software developers to work around IMO, it should be adressed by 3D card manufacturers.

From what I understand, in MSAA modes, both nvidia and ati allowes samples outside the specified texture coordinates.
This seems pretty strange to me, why should this be needed, is it a precision problem?

On the subject of the "blame-game", Valve is right by "blaming" ATI and NVidia IMO.
I dont think we should have to resort to always use "clamp to edge" to be safe from MSAA errors..

Just my 0.2$

stefan
07-22-2003, 02:00 AM
Ostol: What texture coordinates are you using? Do they range from (0,0) - (1,1)? If so, it would be better to use 1/(2N) to 1 - 1/(2N) where N is the width or height of the texture. This prevents OpenGL to sample from outside the texture. Note that this is problematic with mipmaps though.

Zeross
07-22-2003, 04:13 AM
Originally posted by M/\dm/\n:
HMMMM, NV hw has similiar problems in Splinter Cell, though I remember that with oldest drivers, when FSAA wasn't disabled, there was no problems.

BTW, how this texture packing releats to 8x1 4x2 pipe design?


Splinter Cell problem with FSAA is different : it comes from the fact that for applying some effects (nigh vision and stuff like that) it renders the scene in a texture, and Dx8 doesn't support multisampled textures as a render target.

zeckensack
07-22-2003, 06:15 AM
Won,
Yes, that's centroid sampling. You take the mean position of the covered multisampling subpixels as the screen space position for texture lookup. Note that this is the final screen pixel center if all subpixels are covered (ie the polygon interior).
AFAIK centroid is supposed to be app controllable per-texture sampler state, so you'd simply turn it off for overlays.
And yes, my complaint is based on the non-uniform sampling. I guess that makes me a radical http://www.opengl.org/discussion_boards/ubb/wink.gif

Re Epic, I'm certain Tim Sweeney is a clever guy. I'd rather think this is an oversight somewhere in the art toolchain, maybe a custom photoshop plugin created from a 'spoiled' template or something. All U-engine games I've seen share this peculiarity.
Bilinear is not "the best" low pass, but it works. It cannot - by design - fully use the frequency headroom without violating Nyquist, but that's not a problem as long as "when in doubt" the lower detail mipmap level is used. As far as I'm concerned this is how it has always been done.
If you want more detail, there are trilinear and anisotropic filters. These perform worse, of course, but what Epic did, was giving bilinear filtering essentially the same performance characteristics as trilinear filtering while still delivering inferior sampling quality. I don't see how this can be regarded as clever.
If it was a conscious decision to do that, I'll stand by my word, it's stupid.

Given Epic's market position among engine vendors, I feel they could at least apply a little scrutiny and recheck their older code from time to time, especially when it's abundantly clear that performance depends on texture sizes much more than with any other engine. They must have noticed this. It's their responsibility to investigate, plain and simple.

PS: a band pass filter can be regarded as a serial combination of low pass and high pass. It can even be constructed this way (in analog circuitry land) http://www.opengl.org/discussion_boards/ubb/wink.gif
A high pass in front of an ADC serves no purpose but to eliminate DC voltage, ie to center the wave on the zero level http://www.opengl.org/discussion_boards/ubb/wink.gif

Won
07-22-2003, 07:53 AM
zeck --

Amazing...an intelligent conversation.

I'm going to ask my EE friends about this sampling stuff. Here's my thought: I don't think that centroid sampling is that bad of an idea. Basically, it comes down to seeing how significant of a "breach" you'll see in the transfer function. This can be done quantitativly. The thing is, the texture is already being sampled non-uniformly (albeit regularly) because of the way screen pixels lie in texture space. Texture filtering is decided per-fragment, so if you have anisotropic filtering enabled it ought to be able to do the right thing.

I disagree with your assessment of U-engine filtering. It is possible to have superior filtering with the method that Sweeney employs compared to even trilinear. This is because you can chose your filter kernel to be something better or more appropriate than a linear filter while avoiding aliasing. Like I said above, even if you undersample a texture, if the texture is pre-filtered you will not see aliasing. In the extreme case of a single color texture (which has no non-zero freq components) the sampling doesn't matter at all. The signal (texture) bandwidth is independent of the medium's maximum bandwidth (texture resolution). Now for performance reasons, it is clear to me that you don't want to waste resolution so you wouldn't do what Sweeney did. Neither would I, frankly.

I wonder how texture compression factors into this. You can use higher resolution textures and undersample them and still get higher quality filtering for certain images. Intersting.

And to make things even more off-topic:
Yes, of course I guess there needs to be a low-pass filter in there. http://www.opengl.org/discussion_boards/ubb/smile.gif Duh. I was trying to be clever and point out that if you have an ADC that samples at 100kHz, that 100kHz bandwidth doesn't have to include 0Hz. You can sample from 50-150kHz, for example. Obviously, your reconstruction filter has to be the same as your anitaliasing filter. You were right; I guess I wasn't really disagreeing with you, just adding.

-Won

Ostsol
07-22-2003, 11:06 AM
Originally posted by stefan:
Ostol: What texture coordinates are you using? Do they range from (0,0) - (1,1)? If so, it would be better to use 1/(2N) to 1 - 1/(2N) where N is the width or height of the texture. This prevents OpenGL to sample from outside the texture. Note that this is problematic with mipmaps though.
The artifacts also return as the quad gets further and further away from the camera.

I'm just wondering what are the possible uses for texture packing? I know that fonts and animated sprites are some obvious uses, but what else is there?

dorbie
07-22-2003, 02:44 PM
Texture packing can also be useful for using non power of two textures without wasting all the pad space, but mainly it's used simply to reduce state changes (when I've done it in the past).

w.r.t. subsample texel evaluation, it seems obviously flawed in the OpenGL implementation if I understand it correctly, "What the heck were they thinking?". :-)

FWIW with filtering etc, I'd never fully rely on textures being packed exactly adjacent to each other anyway, because you'd experience similar problems even without antialiasing simply due to filtering. It seems that this would only really apply to nearest filters, since others would inevitably bleed in the same way, and even with nearest filters I wouldn't do this because I wouldn't trust the interpolation precision even if hardware did the "right thing". It seems a bit naive to get yourself into this kind of situation in the first place.

The lightmap stuff is only loosely related and applies to filtering and as I've said that's always going to bite you with or without AA.

In the situations that aren't fully self inflicted wounds, I'd say learn your lesson and adjust your coords, current hardware will be around for a long time.

[This message has been edited by dorbie (edited 07-22-2003).]

Coriolis
07-22-2003, 05:35 PM
You don't always need a border pixel in lightmaps; you only need it if bilinear filtering at the edge of the lightmapped polygon could use a pixel not in the lightmap. Sometimes you need two pixels between adjacent lightmaps, sometimes one, and sometimes zero because of this. It also can vary from edge to edge. Not using border pixels when you don't have to can be a significant savings.

M/\dm/\n
07-23-2003, 03:11 AM
DX had problems from its start with pixel adressing => topleft corner/center. PERIOD

Won
07-23-2003, 07:35 AM
This is true, but isn't actually relevant in this case. Even if there isn't a well specified sample location per pixel, for normal D3D rasterization the geometry sample and texture sample come from the same place.

This is not true with the current version of MSAA, whether it is D3D or OpenGL.

-Won

SirKnight
07-23-2003, 07:06 PM
Straight from the horses (Gabe from Valve) mouth: http://www.halflife2.net/forums/showthread.php?s=&threadid=3071

So it sounds like this is not just a HL 2 problem. Although I have never seen artifacts of this kind in any of the games he lists there, but like Gabe said, it shows up a bit more in HL 2 b/c of the lighting variances in their lightmaps.

Also just wondering, lets say you don't use lightmaps but you do pack your regular textures, say some or all of your wood textures in a single large texture, like what's done with lightmaps. Wouldn't you still get similar kinds of artifacts? Or does this only happen with packed lightmaps?

Gabe says there that the solution could be to use centroid sampling, though it will be difficult because of Direct3D not supporting it. Should have used OpenGL. http://www.opengl.org/discussion_boards/ubb/wink.gif Anyway, couldn't you also fix this problem by adding a bit more space inbetween each packed lightmap? Or maybe increase the size of each lightmap by a few pixels so even if it did sample outside, there wouldnt be a problem b/c of the enlarged lightmap. I hope you see what I mean by that.

Anyway, enjoy! http://www.opengl.org/discussion_boards/ubb/smile.gif


-SirKnight

[This message has been edited by SirKnight (edited 07-23-2003).]

dorbie
07-24-2003, 12:37 AM
It only happens where your texture ends and your polygon ends and you rely on exact or near exact sampling for them both to match, AND you don't clamp and aren't tiling and have a nearest filter (since you'd expect this anyway with any other filter).

It's pretty rare that this would impact you, and most of the time when it does it's kinda self inflicted. Most textures are either tiled or clamped or have filters.

Situations where it bites would be packing textures (if you have absolutely no pad or are under heavy minification) or a tiled texture that exactly aligns some image with a polygon edge.
As a 'real world' example think of the interface between a wall and the floor on a tiled texture where the coord is exactly 0 or a multiple of 1 on the edge. It would alias unexpectedly IF you had a nearest filter which would be pretty rare.

Ultimately the solution is to compute the correct texture coordinate for each subsample, but that won't map to what hardware does efficiently. Some more reasonable approach would be to ensure that the sample location is generated at some populated sample location. Presumably what is meant by the centroid is the average location of populated samples. When you consider the potential for multiple anisotropic probes (in addition to the possibility of supersampling with offset aniso probe locations (possibly outmoded now but who knows)) it get's quite tricky IMHO.

I'm still at a bit of a loss to see why this is a big issue with light maps when MIP map filtering would be a bigger issue. Perhaps it's because you can pad and use min LOD to counter the MIP map issue but there's no completely acceptable solution for this supersample issue under extreme minification, since the subpixel displacement for the texture tap increases in scale reguardless of what you do with the filter. It's gotta be something like that.

[This message has been edited by dorbie (edited 07-24-2003).]

MZ
07-24-2003, 07:10 AM
Originally posted by SirKnight:
So it sounds like this is not just a HL 2 problem. Although I have never seen artifacts of this kind in any of the games he lists there (...) ...neither have I, but:
Here are pics I saved many months ago from some HW review site (can't recall which one). I wasn't aware of the AA issue then, but it was surprise to me to notice such big differences in lightmap filtering between different HW (GF4, R9700, Parhelia). Most interesting area is above yellow "x" mark I've placed on the pics.

GF4 (http://www.itstudents.ch/users/dave/files/aa_1_gf410244xaa.jpg)
R9700 (http://www.itstudents.ch/users/dave/files/aa_2_r97006xaajk2.jpg)
Parhelia (http://www.itstudents.ch/users/dave/files/aa_3_parhelia1024_16faa.jpg)

So, 1-texel outline is not enough? (I was under impression Carmack already does it)

edit: pics hosted by unaware davepermen http://www.opengl.org/discussion_boards/ubb/wink.gif


[This message has been edited by MZ (edited 07-24-2003).]

zeckensack
07-24-2003, 01:15 PM
Originally posted by dorbie:
Ultimately the solution is to compute the correct texture coordinate for each subsample, but that won't map to what hardware does efficiently. Some more reasonable approach would be to ensure that the sample location is generated at some populated sample location.I disagree. The texture lookup position for multisampling should be the same as with "AA off", because this is a requirement for making AA application transparent. If the MS texture lookup is not the pixel center, even basic text rendering will start to break. The Geforce 3 had this issue (texture sampling was done on the top left subpixel in 2xRGMSAA), and - surprise - NV25 and up sample on pixel center. You see, NVIDIA deemed it more correct to do it this way, otherwise they wouldn't have changed their hardware. Guess who knows more about texture sampling and AA? Valve or NVIDIA?

This whole "hardware issue" as Valve like to call it is no issue at all. Centroid is simply a means to prevent Valve from shooting themselves in the foot. It solves no problems, it just hides some of them (and in turn leads to new problems).

Baked textures have many issues. This edge sampling thing is only the tip of the iceberg. It's irresponsible to first opt for a technique with a whole array of well known issues, and then proclaiming it's somehow the IHVs' fault when it starts breaking.

If valve went with distinct textures and CLAMP_TO_EDGE (or rather the DXG equivalent) we wouldn't even be discussing this here. The lightmap analogy is just as irresponsible. This is inherently about packed textures, and not about clamp modes. What were they thinking?

[This message has been edited by zeckensack (edited 07-24-2003).]

SirKnight
07-24-2003, 02:09 PM
Ah, thanks dorbie for that explanation, now I understand this issue better.



If valve went with distinct textures and CLAMP_TO_EDGE (or rather the DXG equivalent) we wouldn't even be discussing this here. The lightmap analogy is just as irresponsible. This is inherently about packed textures, and not about clamp modes. What were they thinking?


That's why I always used distinct textues and the clamp_to_edge mode, so I don't have to worry about weird problems like this. I can see why packing textures could help by making things a little faster from not having to change states and even materials as much but doing this now obviously causes other problems to come up. It's just not worth it. Just batch triangles together that shader the same materials/textures/shaders/whatever and send those all at once and all will be well. It's not like doing that is slow, it works well and you don't get the kind of problems like that is currently being discussed.


-SirKnight

dorbie
07-24-2003, 10:20 PM
Zeckensack, while I agree this is largely valve's problem (and they've already said this is a real obscure quality issue), I am a bit more flexible about centroid sampling vs center sampling.

There is a case for *clipping* the filter to the polygon boundary under all circumstances being strictly the 'right thing' TM rather than moving it. I think this is obviously the ideal solution.

Perhaps just moving the samples where the center location is outside the polygon would do as a workaround, it should probably be optional, it's not issue free :-). I understand and agree to an extent with your concerns. It does seem that some people are being whipped into a bit of a lather over what is a design issue that nobody resolves by doing the "right thing" i.e. clip filter to primitive edge (note clamping is just a special case the clipping I mention and doesn't even get it right strictly speaking, but nobody would notice).

The other "right thing", of course is for subsamples to independently generate and fetch texture coordinates at their location. Clipping emulates this exactly without the overhead of doing all that work (clamping wouldn't and needs extra state info etc), how you actually manage to clip a texture filter in hardware is an exercise left for the reader.

[This message has been edited by dorbie (edited 07-24-2003).]

davepermen
07-25-2003, 12:49 AM
zeckensack, you're wrong. the reason is simple: with aa modes the textures get sampled at the wrong places. that IS wrong, hw vendors KNOW this, but they do it because of a very simple reason: performance..

fragment_programs are done once per pixel, too, even with antialiasing. its a simple speed-up, thats why they do it.

but technically its wrong. you should sample at the very point you want to draw the triangle, centroid does that.

behaves like multisampling, a method, where it all works.

you should not blame valve for this issue. aa is not without issues, and that is one of it. and valve is right to cry out loud that this is an issue. while it sounded bigger than it is now, it should still be changeable..

Nutty
07-25-2003, 01:11 AM
A question.

valve's comments said it's fixable via...



2) What are potential solutions?

*Support Centroid Sampling
*Use Pixel Shaders to Clamp Texture Coordinates

Centroid sampling doesn't have the problem that center sampling does in multi-sample antil-aliasing. ATI has supported this form of anti-aliasing for the 9000 series. The tricky part is enabling this when DirectX doesn't easily expose this.


Now, the clamping the texture coordinates in a pixel shader. I dont see how that will stop the MSAA sampling texels "around" this position, and therefore outside of the texture coordinates. Can someone explain to me how clamping in side a pixel shader would fix this?

Also, why doesn't centroid sampling MSAA sample texels outside of the coordinates, and does GL support centroid sampling?

cheers!

dorbie
07-25-2003, 01:35 AM
One of the problems here is it's VERY easy to focus on one individual quality issue and over prioritize it, without considering the possible negative impact. Centroid sampling is an approximation to fix one issue. It's a good one under most circumstances, but it's not 'correct'.

Ideally you want a filter that produces the same sample when antialiasing as when not, just as zekensack says. Consider a fragment on the boundary of two coplanar faces in an AA scene with a tiled texture. Turning AA on for centroid sampling would produce two fragments each sampling texture half a pixel apart and that would effectively convolve the texture along a screen vector orthogonal to the polygon edge. The difference is generally very subtle, but sometimes subtle issues bite you, consider a really dense polygonal textured model, you blur many fragments apparently randomly (for the observer), it would look crap for high frequency textures, you may even see the triangle edges as slightly blurry lines, and zekensack has already pointed out another biggie, textured font rendering along quad diagonals would suck for the reason I've just explained.

You gotta be pragmatic about these things.

Someone says it's 'wrong' not to do this, but clearly it is wrong to do this for other reasons, and that might be important to some applications.

Now there's a grass roots effort forming that expects the "wrong thing" from hardware because of one game. It's a workaround, but it's no more correct and maybe even more wrong that current default behaviour. It's increasingly obvious that zekensack is right and you need to think very carefully about texture filtering to understand why.

FWIW someone from valve should read these last few posts w.r.t. filtering.

[This message has been edited by dorbie (edited 07-25-2003).]

dorbie
07-25-2003, 01:48 AM
Nutty... in reverse order;

centroid sampling AFAIK, samples the texture coordinate adjusted to be located at the centroid of samples *within* the triangle instead of the pixel center, it therefore doesn't touch samples outside the texture because the the polygon edge coincides with the edge of the image you desire to sample.

Clamping in a shader is a workaround because the original problem relates to clamping to a region within a texture holding multiple texture images 'packed' together, and a custom shader can clamp coordinates to a cropped texture region using an additional custom clamp/crop rectangle 'state'.

Nutty
07-25-2003, 01:57 AM
I'm aware about that dorbie, but I thought the problem was the MSAA sampling textures outside the boundry, even when the fragments texture coordinate is inside. Or do you basically just have to pull it back even more as a hack? Wont that have to be AA dependant, i.e. it'll screw up when AA is not used?

dorbie
07-25-2003, 02:03 AM
Nutty, many of the fragments centers are actually outside the polygon and the texture. Look at fragments from a non AA poly, then an AA poly. There are more fragments for the AA poly. All those new fragments failed the original "does the center of this pixel lie within the poly". Naturally for unclamped texture coordinates those same fragments have coordinates outside the desired texture image.

As far as 'pulling it back' The amount you pull it back depends on the scale of the mapping of the texture in screen space over one fragment and that depends on the 3D projection, making simple solutions of limited value under dynamic viewing conditions :-). The simple answer is you need to pull it back one pixel diagonal, the problem is that's an undefined quantity in texture space for the purposes of padding or coordinate trimming.

P.S. I should have probably used "pixels" here instead of "fragments" to make it absolutely clear I'm talking about new extra pixels and not subsamples of any kind.

[This message has been edited by dorbie (edited 07-25-2003).]

zeckensack
07-25-2003, 09:25 PM
Well, I know for a fact that, for instance, Unreal I breaks with multisampling on an R300. I know because I wrote something that would qualify as a 'hacked' rendering backend for this game (and others using the same engine).
The breaking is only apparent on the text overlays and the nature of the issues tells me exactly one thing: the ATI R300 doesn't even get basic multisampling exactly right. The sampling point resolution seems to be restricted to a multiple of one sixth a pixel (which is exactly ATI's MS resolution).

We're talking about borderline luntatic issues here, all of this thread, and we know that not even basic stuff hits the 100 per cent mark on the exact same hardware that Valve claims can be 'cured'. What do we expect? ATI and NVIDIA (and PowerVR, don't forget about them ...) build hardware that fixes Valve's game? Haha.
Ha. Hahaha.
Hohumm.

When I say "lunatic" I mean the sampling issue. Well, baked textures are just full of issues. Some of these we haven't discussed, so let me throw them in.

1)Clamp modes are gone. There's no more GL_REPEAT (or DXG equivalent) anymore. I guess that should be counted as a severe loss for terrain rendering. A mirrored repeat can be emulated, but, in any case, you can't optimize away geometry anymore for static terrain. You need those 'in between' vertices for the tex coords to bounce back and forth. Whatever you save on texture bindings, you'll lose on geometry.

2)You kill mipmapping, beyond a certain amount of minification. Imagine the smallest mipmap level of a baked texture set. You'll have a single texel containing a contribution from every subtexture. You don't want to sample from this smallest level. Ie, you turn off mipmapping at a certain point. Happy shimmering.

dorbie
07-25-2003, 11:15 PM
You've always needed to manage these issues for some applications.

Many people are just lazy when it comes to fixing some of this stuff, or more insidiously have crazy ideas about who is responsible for fixing their rendering problems :-). It's not always up to hardware.

FWIW, the scariest part is *yes* NVIDIA & ATI might just "fix" this in hardware (or the drivers) as the expedient solution. It's very useful to point out the problems with wrong headed proposed solutions that seem sensible at first blush because implementors aren't infallible and obscure quality issues take a back seat to higher profile quality issues.

Valve have already said this is a 'lunatic' issue in their own words. I think the fans kinda missed that though. Of course it's only lunatic when you're getting away without noticing it.

FXO
08-05-2003, 12:29 PM
dorbie:
Im not 100% sure I have understood your post right, but..

1) Doesn't the OpenGL specification say anything about how texcoords should be selected and blended?

2) From what I have understood, this problem doesn't occur when AntiAliasing is turned off, even at high resolutions.


A few thoughts:

AA should only smooth edge-pixels or in some implementations (MSAA?) filter a high resolution image to a smaller one.

I cant see any logic in AA changing the way texels are selected and especially not select pixels out of the specified range.

(yeah, I know the thread is old, sry bout that = )

edit: removed unintended smilies

[This message has been edited by FXO (edited 08-05-2003).]

Adrian
08-05-2003, 01:56 PM
For those who haven't already heard...

<valve|gary> "What about FSAA?"
<valve|gary> Anti-aliased has been fixed on all cards.

Full transcript http://www.hl2db.com/forums/viewtopic.php?p=186#186

FXO
08-06-2003, 01:05 AM
Sorry about the double post.

[This message has been edited by FXO (edited 08-10-2003).]

davepermen
08-06-2003, 01:23 AM
i guess reason is speed.. for textures, filtering "antialiasing" is nicely done with anysotropic filtering.. so they don't need the high sampling rate anymore.. one sample per pixel is enough..

now when they do multiple samples per pixel, and would have to do multiple texture accesses and all per pixel, too, that would mean quite a speed bump.. they would have to run a fragmentprogram several times per pixel, up to 6 times more..

now guess why todays aa is that fast? because they don't.. fragment programs run at one fragment per pixel, no mather how much samples. same for textures (as they are part of the fragment programs..)..

result, a texture sample on the middle of the pixel, while the actual part of the triangle is only on the outer part of the pixel, not in the middle.. result, a wrong sample..


i THINK that is it.. correct me if i'm wrong

Won
08-06-2003, 07:55 AM
Dave is correct. It is essentially the difference between multisampling and supersampling; you can avoid redundant filtering. It works well so long as there's no aliasing issues in your textures or fragment programs. Texture bandwidth and per fragment computation requirements scale with the number of texture samples, while geometry samples only strain the rasterizer which is almost never the bottleneck (less true now with stencil shadows).

-Won

zeckensack
08-06-2003, 08:10 AM
Correct.

Another nice property of multisampling is that all covered fragments have equal color. Even with the most simplistic techniques you can achieve stellar lossless compression ratios for polygon interiors.
Once this is implemented you'll only pay a bandwidth overhead for edges, which make up only a small fraction of a frame. That's why the high AA modes on R300 are so fast.

dave,
I wouldn't say that the 'center of pixel' sampling method is wrong. I've already elaborated on that in this very thread.

davepermen
08-06-2003, 11:29 AM
well, i do understand why they do it.. but on the other hand, every fragment that mathers in the end should contain a screenspace position, and the color, generated with a fragment program and texture samples at this very fragments position.

but so what? nobodies perfect..

Won
08-07-2003, 07:57 AM
Well, center-of-pixel sampling isn't always wrong, but it certainly isn't always right.

Here's an off-the-cuff crazy idea you can employ that might solve these problems. What if you sampled the textures in the same place as the geometry? First off, this sounds like supersampling, but there are some tricks you can perform first. First of all, you can bias the texture accesses to be pixel-sized rather than multisample-sized. For nicely-sampled textures all the multi-samples would come from the same small set of 4-8 or so texels. The coherence would be good enough that you would basically be hitting the texture cache with all these accesses, so the fetch overhead wouldn't be very high compared to just bilinear/trilinear. The other trick is that the texture filtering only has to happen once; each multisample only updates the lerp coefficients. From the sample weights determined by the multisamples, you generate the final texture lookup value, and plug that into your fragment program, which is also only executed once per primitive within the pixel.

Wait a minute...this texture fetching scheme probably ends up looking alot like centroid sampling, although there's a chance that you look at slightly more texture samples in my case, which isn't necessarily a good thing. Also, there might be a texture space vs. screen space perspective-correctness issue. Hmm. Time to do some math.

-Won

dorbie
08-08-2003, 04:38 PM
FXO I'm not sure what your issue with the descriptions is, the basic problem is that with multiple subsamples the center of pixel center sample is used to compute the texture coordinate for all samples (effectively) when algorithmically it shouldn't, and it is really nasty because for partial coverage samples < 50% the pixel center is outside the polygon. Brute force supersampling or scaled rendering gives you a new pixel and therefore a new texture coordinate per subsample. It just doesn't apply the the problem being discussed. Yes ideally you want a new texture sample per subsample, this would fix the problem, several of us have said this, but that is slow and isn't done with efficient AA. Texture sample clipping to aa coverage is equivalent in some ways, it IS "the right thing TM" from an algorithmic point of view. It is ALWAYS correct if it can be implemented. Centroid sampling isn't as I explained, and center sampling isn't always right for the original reasons (see screenshot posted in this thread). Really this is a marginal issue, there's a case for saying center sampling is correct, at least we've highlighted the problems with centroid sampling and explained why it is wrong and quite nasty in many situations. I doubt clip sampling is practical or brute force supersampling or some hybrid probably isn't either (thinking about brute force on dege fragments only), you wind up changing the MIP lod etc, and that has issues for some. It's misguided to call for centroid sampling and start bashing anyone who doesn't support it, so maybe we've done our little bit.

I bailed on this thread a while ago so sorry if I missed your posts & replied late, I'm bailing on it again (it's all been said already AFAIK).

[This message has been edited by dorbie (edited 08-08-2003).]