View Full Version : manual bilinear filter

peterfilm

09-04-2009, 10:46 AM

vec4 texture2D_bilinear(in sampler2D t, in vec2 uv, in vec2 textureSize, in vec2 texelSize)

{

vec4 tl = texture2D(t, uv);

vec4 tr = texture2D(t, uv + vec2(texelSize.x, 0.0));

vec4 bl = texture2D(t, uv + vec2(0.0, texelSize.y));

vec4 br = texture2D(t, uv + vec2(texelSize.x, texelSize.y));

vec2 f = fract( uv * textureSize );

vec4 tA = mix( tl, tr, f.x );

vec4 tB = mix( bl, br, f.x );

return mix( tA, tB, f.y );

}

I'm using this in a vertex shader, sampling from a non-mipped GL_NEAREST filtered texture. It works fine for most textures, but as the uv input parameter gets bigger (or the texture size gets smaller, not sure which), it breaks down (e.g. random stepping in the mesh).

Fair enough, I can live with something that could be a precision error.

BUT, it gives different results to sampling directly using texture2D from a texture with bilinear sampling enabled on it - which works fine, no matter what my uv or texture size is.

My question is, is my bilinear sampling function wrong? Is there another more correct way?

thanks.

pete.

DmitryM

09-04-2009, 11:39 AM

For me your bilinear filter looks correct. Be sure you calculate textureSize & texelSize correctly.

You can post more code and add some screenshots to make the investigation easier.

peterfilm

09-04-2009, 01:10 PM

Thanks for your answer. I guessed it was ok, it makes sense to me, plus I've seen lots of sample code on the web that's the same.

I'm 100% sure the textureSize is correct, and texelSize is calculated as 1.0/textureSize, which I assume is correct...unless I'm missing some border issue you all know about.

I can't really post a screenshot - the only data I've got is quite 'sensitive', and I'd get into trouble posting it on a public forum.

The uv could possibly be quite a mad number, as it's the output of some convoluted logic. It shouldn't matter that it's a mad number really, because the texture has GL_REPEAT texcoord addressing enabled. But, as I say, it works for most texture sizes/uv configurations, and also the built in bilinear sample doesn't have a problem with them. Simply replacing the body of that function with a "return texture2D(t, uv);" with GL_LINEAR filtering enabled on the texture gives correct results every time. I need this manual method as a fallback for cards that don't support bilinear filtering in a vertex shader.

thanks for listening, anyway.

love and kisses,

pete.

...edit...

BTW, I've tried replacing the texture2D's with texture2DLod's, because I realise the vertex shader has no means of determining the correct mipmap to use (even though the texture has no mips, nor does it have a mip filter enabled). Made no difference at all.

peterfilm

09-04-2009, 01:25 PM

if the uv coordinate were so large that, when multiplied by the textureSize, it became a huge number, then I'd lose some needed precision in the fractional part, wouldn't I?

So when I take the fractional part, I'd be getting some truncated number.

Could this be the root of my problem?

If I used mod(uv, 1.0) would that deal with negative uv values?

say, if I did this:

vec2 uv(-10.0, -10.0);

uv = mod(uv, 1.0);

what would be the new value of uv?

DmitryM

09-04-2009, 01:57 PM

From the spec:

genType floor (genType x): Returns a value equal to the nearest integer that is less than or equal to x.

genType fract (genType x): Returns x – floor (x).

genType mod (genType x, float y)

genType mod (genType x, genType y)

Modulus. Returns x – y * floor (x/y).

So, mod(-10.0,1.0) will give you 0.0

About the precision after multiplication:

Assuming you have ~1000 (2^10) texture size.

Float32 has 23 bits mantissa. The multiplication result will have not more than 10 bits before the dot (what you don't need) and not less than 13 bits of precision you need. This is more than enough for determining bilinear coefficients.

P.S. I have no idea what's wrong ATM

peterfilm

09-09-2009, 09:32 AM

Yes, you're correct, thanks.

Tried it anyway, just as a sanity check, and no difference. I still have the problem. It is bizarre.

peterfilm

09-09-2009, 09:54 AM

Well, fixed it.

Not happy with how though.

vec4 texture2D_bilinear(in sampler2D t, in vec2 uv, in vec2 textureSize, in vec2 texelSize)

{

vec2 f = fract( uv * textureSize );

uv += texelSize*0.06; // <--- precision hack (anything higher breaks it)

vec4 tl = texture2D(t, uv);

vec4 tr = texture2D(t, uv + vec2(texelSize.x, 0.0));

vec4 bl = texture2D(t, uv + vec2(0.0, texelSize.y));

vec4 br = texture2D(t, uv + vec2(texelSize.x, texelSize.y));

vec4 tA = mix( tl, tr, f.x );

vec4 tB = mix( bl, br, f.x );

return mix( tA, tB, f.y );

}

Ugly, but it works in all cases now.

It was obviously point sampling the same texels occasionally, so the slope was lost.

Anyone shed any light on why?

NVidia Quadro 5800, driver: 190.38 OS: XP64,SP2

Powered by vBulletin® Version 4.2.5 Copyright © 2018 vBulletin Solutions Inc. All rights reserved.