If you multiply a fraction that isn't a power a number greater than 1 the returned result will be incorrect (for example PICO-8 returns 9.9991 from print(0.1*100)). I'm aware that this is a problem inherent to fixed point numbers, however some special cases would be greatly appreciated.
@Catatafish ah, good point, i was trying to extract digits of a number because tostr() and by extension print() rounds small fractions incorrectly (0.0001 becomes 0 for example) i should probably bring that up in a new report and close this one.
Oh right, I misunderstood the issue, sorry.
Depending on what you're doing with it you could use the hex version of tostr + regular print to see precise values. For non-debug purposes you'll probably have to roll your own.
Can you explain further? It seems part of this conversation got deleted and I'm interested in the details. thanks!
Yeah, sorry for that too!
It wasn't relevant, so I removed it to avoid derailing the thread (and then wished I hadn't as now it looks weird), but the short version is I went off on a diatribe about floating point.
[Please log in to post a comment]