User talk:Numerical investigations

The following is a list of experiments I did to figure out how Petit Computer handles numerical values. Some (most) of these were posted to the wiki before, but I can't figure out if it was deleted or moved somewhere I can't find (if moved, sorry for the duplication).

I can't quite make sense of the results.

?(1/4096)*1000

0.244

?(1/4097)*1000

0

?524287

524287

?-524287

-524287

?524288

Overflow (?)

?-524288

Overflow (?)

?-524287.99

-524287.99

?-524287.99999

-524288

?524287.99

524287.99

?524287.99999

524288

A=524287

OK

?A+(2047/2048)

524288

?A+(4095/4096)

Overflow (?)

A=-524287

?A-(524286/524287)

-524288

?-0

0

A=FLOOR(-524287.1)

OK

?A

-0

?ABS(A)

-0

?A/2

-262144

?A/10

-52428.8

?A*.1

-52352

SquareFingers (talk) 21:16, July 26, 2012 (UTC)