port 6667 is the most commonly used IRC port as it is the default. If you're having difficulties using that port, try some of the alternative ports listed in the server's MOTD.
Lag is almost never due to many users on the same port, a normal server can handle many tousands of users on port 6667. The real reason is most likley the poor connection between the users ISP and the ISP hosting the irc server.
pick a server sponsored by an ISP with better connectivity with your own ISP would be the best choice, (an isp close to yourself such as same country is maybe an easier way to knowing what server to use.)