-32768 and 1 are both perfectly representable in the 16.16 fixed format, as 0x8000.0000 and 0x0001.0000. As far as I can see, this division should be perfect with no floating point shenanigans.
However, 0x8000.0000/0x0001.0000==0x8000.0001
which is throwing a wrench into some code I'm writing.
Other nearby operations work fine, such as
0x8000/2
and
0x7fff.ffff/1
and
0x8000.0001/1
Then there's another group of operations that seem to be technically misbehaving but in a possibly desirable way.
0x4000/0x0.8 0x2000/0x0.4 ... 0x1/0x0.0001
should technically all overflow to 0x8000 (-32768) but they actually evaluate to 0x7fff.ffff which renders as 32768 because it's higher than 32767.9999. Maybe this is intentional, and maybe it's helpful, but I mention it here because it might also be related.
PS: there are markdown rendering bugs affecting this post which is why it's weirdly formatted
Tried it myself:
That's definitely a low-level bug and it's the sort of thing that'll trip me up eventually too, because I tend to work with wonky edge cases a lot, like the outpost (0x8000) value. Hope @zep sees this.
[Please log in to post a comment]