PDA

View Full Version : opengl texture sampling



michagl
03-28-2005, 08:29 AM
for a while i've noticed a disparity between my subpixel sampling and opengls. i think i've decided that i'm doing nothing wrong.

today i think i realized that opengl is sampling pixels from the center rather than the edge. for instance for an identity texture matrix, 0.0 and 1.0 will sample the center of the edge pixels rather than the very edge.

can someone confirm this for me. and assuming this is the case, is there any state i can set so that pixels will be sampled at the corner?

sincerely,

michael

Jan
03-28-2005, 11:11 AM
Yes, i can confirm, that this is the default behaviour. AFAIK it is defined to act like this in the OpenGL spec.

D3D samples from the corner, if i'm not mistaken, and that seems to be a problem in some situations.

Anyway, i donīt think there is an extension to change this behaviour in OpenGL. If you need to sample from the corner, you need to change your texture-coordinates, by subtracting (0.5 / texture-width|height).

Hope that helps.
Jan.

michagl
03-28-2005, 11:20 AM
Originally posted by Jan:
Yes, i can confirm, that this is the default behaviour. AFAIK it is defined to act like this in the OpenGL spec.

D3D samples from the corner, if i'm not mistaken, and that seems to be a problem in some situations.

Anyway, i donīt think there is an extension to change this behaviour in OpenGL. If you need to sample from the corner, you need to change your texture-coordinates, by subtracting (0.5 / texture-width|height).

Hope that helps.
Jan.thanks for confirming my suspicions. i'm still hoping there is a state variable to change this. if anyone knows of such please chime int.

i would think if directx samples from corners there should be someway to augment this behavior in hardware.

its really going to be a headache for me to manage this on the cpu.

sincerely,

michael

knackered
03-28-2005, 11:42 AM
use the texture matrix - easy.

michagl
03-28-2005, 02:47 PM
Originally posted by knackered:
use the texture matrix - easy.a pipeline optimization paper someone here suggested i read said that using the texture matrix significantly cuts into performance. that is apparently the hardware is traditionally optimized for an identity texture matrix.

ZbuffeR
03-28-2005, 04:40 PM
For texel edge sampling, use texture borders.
very simple, instead of a 4*4 rgb array, define a 6*6 one in tables :
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB8, 4, 4, 1, GL_RGB, GL_UNSIGNED_BYTE, tables[i]);
Then clamp like this :
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_S,GL _CLAMP);

Hope it helped.

michagl
03-28-2005, 06:56 PM
Originally posted by ZbuffeR:
For texel edge sampling, use texture borders.
very simple, instead of a 4*4 rgb array, define a 6*6 one in tables :
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB8, 4, 4, 1, GL_RGB, GL_UNSIGNED_BYTE, tables[i]);
Then clamp like this :
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_WRAP_S,GL _CLAMP);

Hope it helped.sorry, but i just don't see how using a bordered texture would force sampling on the corners?

even if somehow it did, it shouldn't be necessary.

can anyone else confirm this?

arekkusu
03-28-2005, 09:08 PM
If your problem is edge texel sampling (you haven't actually said what the problem is), there is state to control that. You can choose from several edge behaviors, read the spec for the core and extensions like ARB_texture_border_clamp and SGIS_texture_edge_clamp.

def
03-28-2005, 11:05 PM
You guys always make me second guess my own code... which is good, I think. ;)
I believe the center sampling is the only reasonable way to do it without causing problems on the very edges.
But what about rectangular textures? They do sample from the edges, right? Thats why CLAMP_TO_EDGE does not work with GL_TEXTURE_RECTANGLE_ARB?
This would suggest that hardware supporting GL_TEXTURE_RECTANGLE_ARB can do both sampling methods.

arekkusu
03-28-2005, 11:41 PM
Read the spec. GL_CLAMP, GL_CLAMP_TO_EDGE, and CLAMP_TO_BORDER are supported. Texture borders are not supported though, so you only get the constant border color.

michagl
03-29-2005, 08:32 AM
Originally posted by arekkusu:
Read the spec. GL_CLAMP, GL_CLAMP_TO_EDGE, and CLAMP_TO_BORDER are supported. Texture borders are not supported though, so you only get the constant border color.these only effect behavior on the border texels as i understand them.

i thought i made it clear what the issue was... Jan seemed to pick it up easy at least.

the problem is simple. if you give opengl a texture coordinate of (0.0,1.0) you don't get the very corner of the map, but rather you get the center of the pixel in the corner of the map. same goes for all other pixels. that is to say that opengl places whole number edges in the center of pixels and not in their corners.

if you are not worried about subpixel sampling, or use this method yourself, you might never notice the difference. but personally i've never used this method, and never seen any third party code that uses this sampling method when generating textures.

it is consistant with the opengl rasterizer... but it is at least awkward enough under enough cases, that opengl should provide a state to change this behavior internally.

from what i believe Jan said, directx does not use opengl's sampling method by default. to me this would either imply that this state can be changed in hardware, and opengl should provide a hook (and probably does) ... or that directx does not use an identity texture matrix by default (which is said to be unoptimized). or maybe Jan is wrong, or i read them wrong.

sincerely,

michael

zeckensack
03-29-2005, 10:00 AM
Originally posted by michagl:
i thought i made it clear what the issue was... Jan seemed to pick it up easy at least.

the problem is simple. if you give opengl a texture coordinate of (0.0,1.0) you don't get the very corner of the map, but rather you get the center of the pixel in the corner of the map.It never depends a single vertex's texture coord, so your example is meaningless. It depends on the interpolated texcoord of the fragment, and that in turn depends on the texture coords that lie on the surface edges and their area, aka magnification or minification "ratio".

same goes for all other pixels. that is to say that opengl places whole number edges in the center of pixels and not in their corners.It doesn't.
OpenGL resolves interpolants at pixel centers. (Viewport) pixel edges lie on whole number positions, pixel centers are 0.5 off. Texcoords are exactly the same.

You might think this doesn't match, but because of the interpolation stuff it actually fits perfectly.
An exact hit on the bottom left texel of a n*m texture map happens for any fragment where the interpolated texcoord is (1/2n;1/2m).

Or to put it into practice, if you draw a 2x2 pixel quad and want an exact mapping of a 2x2 texture, you do this:
it is consistant with the opengl rasterizer... but it is at least awkward enough under enough cases, that opengl should provide a state to change this behavior internally.It is never awkward. It guarantees that surface detail doesn't drift as a result of varying magnification, it guarantees that texturing works independent of viewport sizes and is the only way it can sensibly be specified.

Whatever you're trying to do, just don't. You're only going to break things. The OpenGL way works just fine.

from what i believe Jan said, directx does not use opengl's sampling method by default.Direct3D v5 did what you suggested, and because everyone agreed that it sucked, Direct3D was respecced to follow OpenGL's method.

Go look for an "Alternate pixel center" option hidden somewhere in even current ATI and NVIDIA drivers' control panels. That's what's left of it.

zed
03-29-2005, 11:13 AM
perhaps this is of interest http://www.mindcontrol.org/~hplus/graphics/opengl-pixel-perfect.html

michagl
03-29-2005, 12:18 PM
i don't think you understand zack.

what seems to happen. is if you have for the sake of argument a 4 pixel one dimensional texture (0,1,2,3) and linear filtering is enabled.

if you give opengl the coordinates 0.0 you get the center of pixel 0... if you give it 1.0 you get the center of pixel 3.

so it reasons to follow that if give 0.5 you get the edge between pixel 1 and pixel 2.

so 0.5 is identical, but 0.0, and 1.0 do not hit edges, but hit centers.

now if you do 0.25 and 0.75 you get something not on an edge or center at all.

now i could be wrong, but this seems to be what opengl is doing.

-----

now if i had my way. 0.0 would be the first edge, and 1.0 would be the last edge, and 0.25, 0.5, and 0.75 would all fall on the first edges of pixels (1,2,3) respectively.

this sort of mapping is more natural, and i believe what most people expect.

you won't notice this at all until you are trying to synchronize subpixel sampling with opengl filtering.

-----

now i might have a bug somewhere, but i don't think so. this is what appears to be happening.

zeckensack
03-29-2005, 01:34 PM
Originally posted by michagl:
i don't think you understand zack.

what seems to happen. is if you have for the sake of argument a 4 pixel one dimensional texture (0,1,2,3) and linear filtering is enabled.

if you give opengl the coordinates 0.0 you get the center of pixel 0... if you give it 1.0 you get the center of pixel 3.And you still don't understand interpolation.
I already said it: you don't get anything if you don't specify at least a line with two vertices. And that's the root of your misunderstanding.

If, given your 1D texture, you specify texcoords 0 and 0.25 for a line, you'll hit somewhere on the first texel. "Somewhere" depends on the viewport size of the line. If the line is exactly one pixel long, you'll get exactly the color of the first texel, and in this case it won't even matter if you do linear filtering or point sampling.

Just try the code given above. You can easily modify it to use a 4x4 texture or your 4x1 texture by changing the glViewport parameters appropriately. It just won't matter. As long as the texture has the same dimensions as the viewport, you'll get an exact 1:1 mapping from texel to pixel.

now if i had my way. 0.0 would be the first edge, and 1.0 would be the last edge, and 0.25, 0.5, and 0.75 would all fall on the first edges of pixels (1,2,3) respectively.They do. Try drawing a quad where all vertices have the same texcoords, and see for yourself.

michagl
03-29-2005, 07:24 PM
Originally posted by zeckensack:
They do. Try drawing a quad where all vertices have the same texcoords, and see for yourself.well if you are right, you are in direct opposition to Jan's original post which references the specifications.

i will check for bugs in my code again as soon as my computer finishes a task... probably tomorrow.

basicly i'm building a height field... the colour map is lower resolution the displacement map. my displacements are off by about a half of a pixel in the y axis, and maybe in the x axis (i haven't really payed attention to the x axis).

i see something that might make a difference. i will check it asap and get back.

michagl
03-30-2005, 08:19 AM
i made a small change which seems to help my situation. not sure if it is perfect, synchronized with opengl, or just counter acting opengl.

sampling a texture in memory is actually pretty tricky.

can someone post some tried and true buffer sampling c pseudo-code for image widthxheightxchannels byte encoded.

also just curious? can opengl upload interleaved and uninterleaved channels? i figure 'PixelTransfer' would be the operative command. what is more common and useful? storing images by the channel or by the element?

and i'm still interested in just how opengl samples textures. there appears to be quite conflicting ideas on this matter within this thread.

sincerely,

michael

michagl
03-30-2005, 09:03 AM
oops... no that change i made causes a fundamental 'bug'. i was sure i had tried that before.

i'm not going to give up till i get that straight.

later i think i will try using the texture matrix to see if it can straighten out the situation.

i really don't think the texture matrix is a good fix though. is the texture matrix automaticly available in shaders lilke i believe the modelview and inverse modelview are supposed to be?

Aeluned
03-30-2005, 09:30 AM
in GLSL the texture matrix is available with
gl_TextureMatrix[i], where i is the texture unit.



and i'm still interested in just how opengl samples textures. there appears to be quite conflicting ideas on this matter within this thread.

...you could always read the spec.

AFAIK texels are sampled from the center, not the edge.

michagl
04-01-2005, 06:57 AM
Originally posted by Aeluned:
in GLSL the texture matrix is available with
gl_TextureMatrix[i], where i is the texture unit.



and i'm still interested in just how opengl samples textures. there appears to be quite conflicting ideas on this matter within this thread.

...you could always read the spec.

AFAIK texels are sampled from the center, not the edge.i will be away for a little while i'm sorry to say.

yes, i think i may've confused some people at some point when mixing the terms 'texels' and 'pixels' up. later i realized that this might be a way to diffrentiate between texture pixels and screen space pixels.

finally i don't understand 'gl_TextureMatrix[i]'.

is that accessing a single element of the texture matrix?

for cg for instance would mul(gl_TextureMatrix,texcoords) work?

i understand that the modelview and inverse modelview are automaticly accessible as constants. that is they are always up to date and do not use up 'constant' memory? assuming this is correct, is the texture matrix the same way. and what sort of matrix is the texture matrix? 4x4 i figure. is there anyway to use a 2 dimensional texture matrix to save cycles?

as for reading core opengl specs... i think i might try to start with opengl2.0.

Aeluned
04-01-2005, 09:02 AM
yeah, gl_TextureMatrix[] is an array of texture
matrices with a size of the available number of
texture units.

you can query this value using
GL_MAX_TEXTURES_UNITS i believe, but I've never
seen more than 4 units exposed to an application.

not sure about Cg, i don't use it.



and what sort of matrix is the texture matrix? 4x4 i figure. is there anyway to use a 2 dimensional texture matrix to save cycles?
You're always fighting what's already in
place :p

How exactly would that save cycles?
This stuff is done in hardware using 4x4
homogenous matrices.
I also think it would break affine transformations.

I'm pretty sure there's a reason things are in
place the way they are.
Anyway, I don't know much about using a 2x2
matrix.

michagl
04-02-2005, 12:15 PM
Originally posted by Aeluned:
yeah, gl_TextureMatrix[] is an array of texture
matrices with a size of the available number of
texture units.

you can query this value using
GL_MAX_TEXTURES_UNITS i believe, but I've never
seen more than 4 units exposed to an application.

not sure about Cg, i don't use it.



and what sort of matrix is the texture matrix? 4x4 i figure. is there anyway to use a 2 dimensional texture matrix to save cycles?
You're always fighting what's already in
place :p

How exactly would that save cycles?
This stuff is done in hardware using 4x4
homogenous matrices.
I also think it would break affine transformations.

I'm pretty sure there's a reason things are in
place the way they are.
Anyway, I don't know much about using a 2x2
matrix.oh yeah.... multitexture eh. seems silly to use the texture matrices just to get around opengl's loopy sampling. do game artist adjust their texture coordinates so they hit on the center i wonder? it seems just to much to ask without providing an alternative method. especially because the centers change depending on the width and height of the texture i believe.

as for 3x3 vs 4x4, support is built into the high level language shaders, but i guess it is just more for storage than anything else. if people actually used texture transforms regularly and were not penalized for doing so, it might be worth providing hardware for 3x3 (2d affine) matrix transform... the same hardware could be used for non translatable matrices as well. anyhow, it never hurts to ask.

this issue is really burning. me i don't want to have to have an internal texture coordinate format, i wan't coordinates to be transparent, and i don't want end users to have to worry about this issue. its a pain in the ass really. even if i used the texture matrix to side step... two versions of custom shaders would need be maintaned depending on whether the matrix is used or not.

i'm really disappointed with this. its a lose lose situation no matter how i go about it. center sampling is just weird for software subsampling.

i'm pretty sure i know what i have to do. setting it up will just be a pain though. i still don't see what the benefit of center sampling is. numericly its all cockeyed. why can't people just agree on an approach. and the burden of the approach should be put on the hardware, and not end users.

zeckensack
04-02-2005, 12:54 PM
i'm really disappointed with this. its a lose lose situation no matter how i go about it. center sampling is just weird for software subsampling.

i'm pretty sure i know what i have to do. setting it up will just be a pain though. i still don't see what the benefit of center sampling is. numericly its all cockeyed. why can't people just agree on an approach. and the burden of the approach should be put on the hardware, and not end users.Don't "set up" anything!
If you're trying to be clever about this it only shows that you didn't understand it. Really, don't do anything. Use naive texture coordinates and everything will just work. OTOH try to be clever and something will break.

Can you actually show an example of this "issue" in action or did you arrive at your conclusion (GL texture sampling = bad) through pure theory?

Unless you can provide some shots, or code, or just anything that we as your humble audience can verify, this feels like nothing more than a huge waste of time. For all I know, your "issue" may simply be a matter of CLAMP_TO_EDGE vs CLAMP and not related to texture coords at all. There's just no substance here to merit all this lamentation. Until some comes up, I hereby tip my head and withdraw from this "discussion".

Re benefits: it just works, independent of texture size and aspect ratio, object size and viewport size. Demonstrate the opposite if you can.

knackered
04-02-2005, 02:19 PM
This is typical of michagl.
How much time would it take for you to just read the GL spec? All this waffling, pontificating, and procrastination has taken so much more time than to simply read the spec....and that would have just involved one person reading, not a whole bunch arguing.

michagl
04-02-2005, 02:42 PM
everyone appears to agree that the specs say opengl samples at the center. and i could see how this could be useful for maybe rendering to a texture where it is only logical that opengl should sample its rasterization targets just like it would in screen space.

but no one thinks about textures this way in reality. and in my experience modeling environments like maya and what have you don't reflect this fact conceptually.

simply put this gets by because you will never notice if you are not looking at the texture at the subpixel level.

yes this is very much a problem, but it looks like i will just have to learn to live with it.

to see the effects yourself, just setup a heightmap which uses subpixel sampling to fill in vertices between the pixels. your height values will not match opengls colour map for instance using the same texture coordinates... unless you are doing opengl style sampling in your app, which as described is much more convoluted, in the fact that whole numbers never reliably fall on texel edges or centers. this makes mapping to a discrete buffer a pain.

for instance for a 10x10 heightmap, values like 0.1, 0.2, 0.3, 0.4 for the most part would neither fall on edges or centers.

opengl appears to base its mapping on setting 0.0 and 0.1 for identity texture matrix, to the center of the corner pixels... everything inbetween is just a matter of interpolation.

since no one else seems to care. just don't respond.

this seems like a big deal to me though. and as at least a couple of people have stated. they were not previosly thinking of texel sampling in these terms.

if you don't care about your filtering being off by a half a texel. then go on your marry way. otherwise you should be thinking about modifying your systems.

if you are using a high resolution displacement map, and a low resolution colour map. this disparity can be ignormous. for instance i've worked with a 10000 width height field, and a 512 colour map. the colour map can be off by 100's of kilometers. oceans can land on hills. its pretty ugly stuff.

there should be an open debate on this rather than a couple staunch actors just saying 'sit down and shut up'.

michagl
04-02-2005, 02:57 PM
i read this in a link i think 'zed' originally suggested. i don't know why i forgot to look at that link. anyhow:

"Second, you should realize that "0,0" and "1,1" is the same texture coordinate, and sits right between all four corners of the texture, when you're using REPEAT (and 0,0 is in the low-left corner of the low-left texel when using CLAMP). To sample the center of a texel, in a texture with side length L, you need to have a coordinate that is (0.5/L)+(N/L) where N is an integer."

this appears to reject the possiblity that opengl samples on centers as far as this is true.

i have to wonder though what kind of relatively complex algorithm the hardware would use to decide which directin to interpolate through if it is given an edge from 0.0 to 1.0 if in fact they are the same singularity.

perhaps the quoted statement is not true, and opengl/hardware relies on the fact that 0.0 and 1.0 sample the centers of corner pixels just so that it can avoid ambiguity.

any thoughts?

i've never denied that my subsampling code may not be quite right. but i've stared at it for at least a couple hours, and have looked at many other third party examples of software texture sampling and texture generation.

michagl
04-02-2005, 03:09 PM
i give up honestly... i think i will just ignore the problem and hope it goes away. hopefully some day i will notice a logical error in my code.

i apreciate your concerns.

sincerely,

michael

V-man
04-02-2005, 08:42 PM
"Second, you should realize that "0,0" and "1,1" is the same texture coordinate, and sits right between all four corners of the texture, when you're using REPEAT (and 0,0 is in the low-left corner of the low-left texel when using CLAMP). To sample the center of a texel, in a texture with side length L, you need to have a coordinate that is (0.5/L)+(N/L) where N is an integer."

this appears to reject the possiblity that opengl samples on centers as far as this is true.

i have to wonder though what kind of relatively complex algorithm the hardware would use to decide which directin to interpolate through if it is given an edge from 0.0 to 1.0 if in fact they are the same singularity. The equation has this component "(0.5/L)" because GL samples at the center of texels.

I think you are having trouble understanding texture coordinate addressing.
0.0 and 1.0 are special cases and yes, the hw has the appropriate circuit that handles sampling depending on the texcoord addressing state.

What problem are you trying to solve? How do you setup your texture?


a pipeline optimization paper someone here suggested i read said that using the texture matrix significantly cuts into performance. that is apparently the hardware is traditionally optimized for an identity texture matrix. The vertex stage is typically not the bottleneck. I would not worry about 4 DP4 operations per tex stage.

def
04-03-2005, 01:11 AM
Originally posted by michagl:
...for instance for a 10x10 heightmap, values like 0.1, 0.2, 0.3, 0.4 for the most part would neither fall on edges or centers.That is just what does happen! 0.1 lies exactly on the edge between the first and second texel.


...if you are using a high resolution displacement map, and a low resolution colour map. this disparity can be ignormous. for instance i've worked with a 10000 width height field, and a 512 colour map. the colour map can be off by 100's of kilometers. oceans can land on hills. its pretty ugly stuff.Right. When sampling different resolutions the only way to do that (even without knowing the dimensions) is the way it is implemented in OpenGL. The square height field and the texture match edge to edge (0.0 - 1.0), so if you sample 0.25 in u from both, you get exactly the position one 4th across its width.
If this wouldn't work, mipmapping would not work either.

Overmind
04-03-2005, 02:26 PM
michagl, the other posters are not trying to argue with you if the OpenGL way is good or bad, they are trying to tell you that your assumption about 0.0 being the center of a texel is wrong.

I think you misunderstand a crucial fact:
OpenGL does sample on the center of the fragment, but that does not mean it will be the center of a texel. This has nothing to do with what you get for a particular texture coordinate, but with what texture coordinate is used in the lookup.

Lets just review the 1D case of a horizontal line for simplicity:

The texture coordinate 0.0 is the left edge of the first texel.
Now you draw a line, with a texture coordinate of 0.0 at the first vertex. OpenGL generates fragments from this line, and the left edge of the first fragment will have the texture coordinate 0.0.
But OpenGL samples at the center of fragments, so the texture coordinate of the (center of the) fragment will be 0.0 + half a pixel.
Now if for some reason pixels and texels are the same size, 0.0 + half a pixel will be the same as 0.0 + half a texel, which is the center of the first texel.

So you will get the center of the first texel on the first pixel, but not because 0.0 is the center. It's because the lookup didn't occur at 0.0, but at 0.0 + some small value...

You can easily verify this. Just draw a quad with texture coordinate 0.0 on every vertex, and clamp to border. This way the pixel size is 0 in texture coordinate space, so you will really get exactly 0.0 (== 0.0 + half a pixel ;) ) as texture coordinate on every fragment. This should get you a 50% mix of the border color and the first texel, proving that 0.0 is actually the edge of a texel.

michagl
04-05-2005, 03:49 PM
like i said before i apreciate the input. but there is just too much info here and apparently conclifting possibilities for me to wrap my head around.

for what its worth, i never had any issue with 'fragments', or rasterization. its just the texel sampling which doesn't seem to be working out right for me.

someone asked how i am setting up matters. basicly i have a mesh, with texture coordinates. i'm displacing the vertices in the normal direction based on a filtered sampling of a heightmap done in a preprocessing step.

to arrive at my displacement sample. i take the texture coordinate of the vertex, chop off the fractional component, and pick the pixel that lies on the edge of the whole component. then i build a 3x3 kernel around that pixel, and use the fractional component to interpolate between the center and surrounding pixels.

after everything is done i render the mesh with a colour texture that matches the displacement map pixel for pixel.

from there it appears that the blended subpixel colours produced by the opengl linear mag filter do not align with the displaced vertices. that is there is about a half a pixel disparity between the corresponding colours and elevations.