I've never had this problem, but I also don't visit a lot of different networks. I'm usually only on 2 (undernet and irchighway) and I have a very stable connection as well.

Invision Support
#Invision on irc.irchighway.net