That depends what network you're on.

If you've registered your channel with a network service (e.g. ChanServ, Q, X) you should be able to get it to invite you to the channel. You can then unset the mode normally.