Try disabling (for the time being) any firewall software you may be running. I found running these often an inbound packet would be blocked, causing one side of the connection to stall as it had lost sequence.
If this proves to be the case you may need to update the firewall or add some less stringent rules for the IRC server. I could be totally off track though.
This could also be an MTU issue with your router, however these usually become evident when you initially connect (as at that stage you are sent a lot of burst data from motd, lusers, numerics, autojoins etc.).
Hope that helps