No, this is impossible with TCP. The connection is routed through a specific IP/port pair (in addition to the existing TCP route that was established via handshake). You can't change that in the middle of a session. The same thing would happen for any persistent TCP connection on your system. The difference is that most of your other TCP connections (like your browser or mail app) can disconnect/reconnect without issue, since they are not persistent connections. IRC uses the same connection for the duration of the session.
In short: don't turn off your VPN when it's in use. That's equivalent to pulling your cable/dsl out of the jack in your wall.
