PDA

View Full Version : Problem with integer div/mod



cameni
08-10-2011, 11:41 AM
Found an interesting bug that manifests both on AMD and Nvidia, albeit differently. Expression:



//int v=2;
abs(2047483673*v) % 16807

executed in a fragment shader results in the following values (for int v=2):

CPU: 13457
Nvidia: -13457 ('v' a uniform), 13457 ('v' a constant, compile-time evaluation)
AMD 4xxx: 13460 ('v' a uniform), 13456 ('v' a constant, compile-time evaluation)
AMD 5xxx: 13456 ('v' a uniform), 13456

On Nvidia apparently the abs() in there brings the problem forward, without it it works fine (in GLSL the result of integer modulo is undefined for negative operands).
On AMD it looks like it's being approximated somehow.

I've discovered it while making a randomizer algorithm, and getting different results everywhere.

Since no way of reporting bugs to Nvidia works for me, even though I'm a registered developer there, I'm putting it here.

Alfonse Reinheart
08-10-2011, 12:26 PM
I fail to see how this is a bug (for GLSL). You're overflowing a signed-integer. And since integer literals are signed unless you make them unsigned, you're getting an overflow. What happens in the case of an overflow is undefined, and therefore any answer is legitimate.

cameni
08-10-2011, 12:42 PM
4.1.3 Integers
Operations resulting in overflow or underflow will not cause any exception, nor will they saturate, rather they will “wrap” to yield
the low-order 32 bits of the result.