Posted By: maroon

## $base fraction rounding - 13/09/20 12:35 AM

A couple issues with $base rounding fractions:

(1) $base doesn't correctly handle fractions longer than 6 digits, if rounding to 6 digits causes it to round up to the next higher integer.

//var %i 1 , %a | while (%i isnum 1-99) { var %a %a $base(%i $+ .9999995,10,10,3) | inc %i } | echo -a %a

result:

001.999999 002.999999 003.999999 004.A 005.A 006.A 007.A 008.A 009.A 010.A 011.A 012.A 013.A 014.A 015.A 016.999999 017.999999 018.999999 019.999999 020.999999 021.999999 022.999999 023.999999 024.999999 025.999999 026.999999 027.999999 028.999999 029.999999 030.999999 031.999999 032.A 033.A 034.A 035.A 036.A 037.A 038.A 039.A 040.A 041.A 042.A 043.A 044.A 045.A 046.A 047.A 048.A 049.A 050.A 051.A 052.A 053.A 054.A 055.A 056.A 057.A 058.A 059.A 060.A 061.A 062.A 063.A 064.A 065.A 066.A 067.A 068.A 069.A 070.A 071.A 072.A 073.A 074.A 075.A 076.A 077.A 078.A 079.A 080.A 081.A 082.A 083.A 084.A 085.A 086.A 087.A 088.A 089.A 090.A 091.A 092.A 093.A 094.A 095.A 096.A 097.A 098.A 099.A

Some of those numbers rounded down 'correctly', but that's because of the known float rounding bug that's seen in $round where some numbers incorrectly round the '5' down instead of up. If that last '5' digit is changed to be anything in [6789] then all the above fractions will be '.A'

//var %i .05 , %a | while (%i < 99) { var %a %a $round(%i,1) | inc %i } | echo -a %a

result: 0.1 1.1 2 3 4 5 6 7 8.1 9.1 10.1 11.1 12.1 13.1 14.1 15.1 16.1 17.1 18.1 19.1 20.1 21.1 22.1 23.1 24.1 25.1 26.1 27.1 28.1 29.1 30.1 31.1 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98

//var -s %price 1.00 , %tax .095 , %total %price + %tax | echo -a %price + tax %tax = $ $+ $round(%total,2)

result: 1.00 + tax .095 = $1.09

(2) At higher outbase, $base also shortens the fraction to fewer digits than needed.

//var -s %a $base(9.999999,10,11) , %b $base(%a,11,10)

result: 9.999999

//var -s %a $base(9.999999,10,36) , %b $base(%a,36,10)

result: 9

//echo -a $calc( $log($calc(10^6)) / $log(2) ) vs $calc( $log($calc(36^3)) / $log(2) )

result: 19.931574 vs 15.509779

The above shows that 6 base10 digits fits within 20 bits, while 3 base36 digits fits within 16 bits, yet base36 won't output fractions longer than 2 digits.

The base-X fraction should be long enough to translate a fraction from base-10 to base-X, then translate back to a base-10 fraction reasonably close to the original fraction.

//var %out 36 , %a $base($pi,10,%out) , %b $base(%a,%out,10) | echo -a pi is %b

result: pi is 3.13966

Bases up through 28 will output 6 digit fractions, at which point it requires 29 bits to hold that fraction. Above base28, the length drastically drops until bases 35-36 will not output fractions longer than 2 digits.

//var %i 1.654321 , %out 36 , %a 0 0 0 0 0 0 0 | while (%i < 10) { var %n $len($gettok($base(%i,10,%out),2,46)) + 1 , %t 1 + $gettok(%a,%n,32) , %a $puttok(%a,%t,%n,32) | inc %i .01 } | echo -a %out : $regsubex(foo,%a,/(\d+)/g,$+($calc(\n -1),:,\t))

result:

28 : 0:0 1:0 2:0 3:0 4:0 5:33 6:802

29 : 0:0 1:0 2:0 3:9 4:25 5:801 6:0

30 : 0:0 1:0 2:0 3:0 4:0 5:835 6:0

31 : 0:0 1:0 2:0 3:33 4:802 5:0 6:0

32 : 0:0 1:0 2:0 3:33 4:802 5:0 6:0

33 : 0:0 1:0 2:26 3:809 4:0 5:0 6:0

34 : 0:0 1:0 2:34 3:801 4:0 5:0 6:0

35 : 0:8 1:33 2:794 3:0 4:0 5:0 6:0

36 : 0:8 1:25 2:802 3:0 4:0 5:0 6:0

(1) $base doesn't correctly handle fractions longer than 6 digits, if rounding to 6 digits causes it to round up to the next higher integer.

//var %i 1 , %a | while (%i isnum 1-99) { var %a %a $base(%i $+ .9999995,10,10,3) | inc %i } | echo -a %a

result:

001.999999 002.999999 003.999999 004.A 005.A 006.A 007.A 008.A 009.A 010.A 011.A 012.A 013.A 014.A 015.A 016.999999 017.999999 018.999999 019.999999 020.999999 021.999999 022.999999 023.999999 024.999999 025.999999 026.999999 027.999999 028.999999 029.999999 030.999999 031.999999 032.A 033.A 034.A 035.A 036.A 037.A 038.A 039.A 040.A 041.A 042.A 043.A 044.A 045.A 046.A 047.A 048.A 049.A 050.A 051.A 052.A 053.A 054.A 055.A 056.A 057.A 058.A 059.A 060.A 061.A 062.A 063.A 064.A 065.A 066.A 067.A 068.A 069.A 070.A 071.A 072.A 073.A 074.A 075.A 076.A 077.A 078.A 079.A 080.A 081.A 082.A 083.A 084.A 085.A 086.A 087.A 088.A 089.A 090.A 091.A 092.A 093.A 094.A 095.A 096.A 097.A 098.A 099.A

Some of those numbers rounded down 'correctly', but that's because of the known float rounding bug that's seen in $round where some numbers incorrectly round the '5' down instead of up. If that last '5' digit is changed to be anything in [6789] then all the above fractions will be '.A'

//var %i .05 , %a | while (%i < 99) { var %a %a $round(%i,1) | inc %i } | echo -a %a

result: 0.1 1.1 2 3 4 5 6 7 8.1 9.1 10.1 11.1 12.1 13.1 14.1 15.1 16.1 17.1 18.1 19.1 20.1 21.1 22.1 23.1 24.1 25.1 26.1 27.1 28.1 29.1 30.1 31.1 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98

//var -s %price 1.00 , %tax .095 , %total %price + %tax | echo -a %price + tax %tax = $ $+ $round(%total,2)

result: 1.00 + tax .095 = $1.09

(2) At higher outbase, $base also shortens the fraction to fewer digits than needed.

//var -s %a $base(9.999999,10,11) , %b $base(%a,11,10)

result: 9.999999

//var -s %a $base(9.999999,10,36) , %b $base(%a,36,10)

result: 9

//echo -a $calc( $log($calc(10^6)) / $log(2) ) vs $calc( $log($calc(36^3)) / $log(2) )

result: 19.931574 vs 15.509779

The above shows that 6 base10 digits fits within 20 bits, while 3 base36 digits fits within 16 bits, yet base36 won't output fractions longer than 2 digits.

The base-X fraction should be long enough to translate a fraction from base-10 to base-X, then translate back to a base-10 fraction reasonably close to the original fraction.

//var %out 36 , %a $base($pi,10,%out) , %b $base(%a,%out,10) | echo -a pi is %b

result: pi is 3.13966

Bases up through 28 will output 6 digit fractions, at which point it requires 29 bits to hold that fraction. Above base28, the length drastically drops until bases 35-36 will not output fractions longer than 2 digits.

//var %i 1.654321 , %out 36 , %a 0 0 0 0 0 0 0 | while (%i < 10) { var %n $len($gettok($base(%i,10,%out),2,46)) + 1 , %t 1 + $gettok(%a,%n,32) , %a $puttok(%a,%t,%n,32) | inc %i .01 } | echo -a %out : $regsubex(foo,%a,/(\d+)/g,$+($calc(\n -1),:,\t))

result:

28 : 0:0 1:0 2:0 3:0 4:0 5:33 6:802

29 : 0:0 1:0 2:0 3:9 4:25 5:801 6:0

30 : 0:0 1:0 2:0 3:0 4:0 5:835 6:0

31 : 0:0 1:0 2:0 3:33 4:802 5:0 6:0

32 : 0:0 1:0 2:0 3:33 4:802 5:0 6:0

33 : 0:0 1:0 2:26 3:809 4:0 5:0 6:0

34 : 0:0 1:0 2:34 3:801 4:0 5:0 6:0

35 : 0:8 1:33 2:794 3:0 4:0 5:0 6:0

36 : 0:8 1:25 2:802 3:0 4:0 5:0 6:0