Hi, i'm new to coding in general, but i can't understand why some of the arithmetic operations return not exact value.
printh(1 * 0.1) -- 0.1 printh(-1 * 0.1) -- -0.09998 |
How do i fix this without rounding? Thanks
This is a common quirk with floating and fixed point arithmetic (Pico-8 numbers use fixed point) and stems from the fact that certain fractions can not be exactly represented and will instead be rounded to the nearest possible value.
In Pico-8, a number is stored in 32 bits: 16 bits are used to represent the whole number (from -32768 to 32767) and the remaining 16 are used to represent the fraction as a fraction of 65536 (from 0/65536 to 65535/65536). So if the value 32768 is stored in the fractional part of our number, we get 32768/65536 = 0.5.
This should explain why you get these inaccuracies: numbers that can not be represented as a fraction of 65536 will be rounded up or down. Numbers like 0.5 and 0.25 can be exactly represented, but if we take a number like 0.1, the faction that lies the closest is is 6554/65536 = 0.100006103515625. The inaccuracy is negligible most of the time, but you have to be extra careful when doing exact comparisons between these numbers as, say, 0.100006103515625 * 5 = 0.500030517578125 is not exactly 0.5 (just like how if you were to round 1/3 to 0.33 and do 0.33 + 0.33 + 0.33 = 0.99, which is not exactly 1).
In short, it's just how arithmetic like this works and it's something you will have to live with. It's usually not a problem but you should be a bit careful when doing exact comparisons.
Edit: I forgot to say, when Pico-8 outputs a number to the screen or console, it's usually rounded to the nearest "nice-looking" number, which is why it might tell you that your number is 0.1 when in actuality it is just slightly more or less than 0.1.
I went through this sometime ago with the team at BlitzMAX. Was getting inaccurate results. Let's see how PICO stacks up to the "test."
n=0 cls() for i=0,3,.01 do print(i) n+=1 if n==19 then n=0 repeat flip() until btnp(4) cls() end end |
As you can see, REALLY inaccurate all over the map. In fact the only language I came across that showed any decent accuracy with Real numbers was before Windows time. It was called QBasic 4.5. :)
[Please log in to post a comment]