When it really does it then something is indeed wrong there...
For me it works ok though... it remembers used ports for every entry like you can see in servers.ini as well
[recent]
n0=EFnet: Random serverSERVER:irc.efnet.net:6668:GROUP:EFnet
n1=EFnet: Random serverSERVER:irc.efnet.net:6667GROUP:EFnet
n2=irc.undernet.orgSERVER:irc.undernet.org:7000:GROUP:
n3=DALnet: Random serverSERVER:irc.dal.net:6669:GROUP:DALnet
