Thanks for your bug report. The quirks of floating point calculations have been discussed many times.
See
here and
here and
here for an explanation.
This means that when rounding, for example, N.5, where N is a value, it will for some Ns round up and for others round down.
Apart from that, $base() involves calls to combinations of mod(), round(), pow(), among other calls, so the above behaviour will be compounded with other rounding errors.
Note that if you are using floating point in $base(), there is no guarantee that you will be able to convert back and forth between results. Not all floating point values in one base can be represented in a different base.
As for decimal precision being shortened as the base is increased: ideally, the scripters who requested this would comment at this point, however as it was implemented 17 years ago... I can only assume it was due to issues with precision, overflow, etc. The $base() code has been tweaked so many times since then that this limitation may not even be needed any more.
I am going to remove the precision limit on large bases in the next beta but this will need testing.
