Problem with integer div/mod

Found an interesting bug that manifests both on AMD and Nvidia, albeit differently. Expression:


//int v=2;
  abs(2047483673*v) % 16807

executed in a fragment shader results in the following values (for int v=2):

CPU: 13457
Nvidia: -13457 (‘v’ a uniform), 13457 (‘v’ a constant, compile-time evaluation)
AMD 4xxx: 13460 (‘v’ a uniform), 13456 (‘v’ a constant, compile-time evaluation)
AMD 5xxx: 13456 (‘v’ a uniform), 13456

On Nvidia apparently the abs() in there brings the problem forward, without it it works fine (in GLSL the result of integer modulo is undefined for negative operands).
On AMD it looks like it’s being approximated somehow.

I’ve discovered it while making a randomizer algorithm, and getting different results everywhere.

Since no way of reporting bugs to Nvidia works for me, even though I’m a registered developer there, I’m putting it here.

I fail to see how this is a bug (for GLSL). You’re overflowing a signed-integer. And since integer literals are signed unless you make them unsigned, you’re getting an overflow. What happens in the case of an overflow is undefined, and therefore any answer is legitimate.

4.1.3 Integers
Operations resulting in overflow or underflow will not cause any exception, nor will they saturate, rather they will “wrap” to yield
the low-order 32 bits of the result.

This topic was automatically closed 183 days after the last reply. New replies are no longer allowed.