Register Log In

Forums Developers Precise integer range

Print Thread

Precise integer range #150910 10/06/06 07:19 PM
Joined: Sep 2003 Posts: 38 North America O Olathe OP Ameglian cow
OP Olathe Ameglian cow O Joined: Sep 2003 Posts: 38 North America	If I don't want any precision or rounding errors in $calc, what is the range of integers I can use ?

Re: Precise integer range #150911 10/06/06 08:49 PM
Joined: Sep 2003 Posts: 4,230 D DaveC Hoopy frood
DaveC Hoopy frood D Joined: Sep 2003 Posts: 4,230	I have recieved $calc errors at 16 digit integers, havent noticed any below that yet, so i would say it MAYBE 1-999,999,999,999,999 an example of an error at 16 digits is... //echo -a $calc(9999999999999999 / 9) 1111111111111110.9 * there may be others below this size, i havent found one myself (havent looked hard either) *

Re: Precise integer range #150912 11/06/06 12:03 AM
Joined: Apr 2004 Posts: 871 The Netherlands Sat Hoopy frood
Sat Hoopy frood Joined: Apr 2004 Posts: 871 The Netherlands	-9007199254740992 to +9007199254740992 (-(2^53) to 2^53) Saturn, QuakeNet staff

Link Copied to Clipboard