Yes, I was referring to authentication. Without authentication, there is no way for you to know whether your communication on IRC is being monitored. Since it would be trivial for an IRC server to monitor all messages and to automatically initiate, without any human intervention, MITM attacks during key exchange and then decrypt all messages on-the-fly, it seems to me that this would give users a false sense of security. As you say, you would need to exchange keys on a secure channel outside of IRC, which makes things a little more complicated.