i'd still try what i suggested first in the last post. Beyond that, some of these things have helped ppl who posted here with the same slow down/stop prob:
-check router firmware/software
-if you (or invision) have any of those download speed patch booster things, get rid of them and fix whatever they edited in the registry
-disable xp firewall
-try connecting to a diff server port, 6667 seems to give some ppl probs like this
-some have found it related to some nic cards as i recall (try the search feature)
it could also be something at isp end, some say they are using "interference techniques to stop bandwidth abuse".
and of course, it could be on the other persons end.