IRC servers place their own limit on the allowed length of a message. Usually the maximum length of the whole server message is around 512 characters, which includes the PRIVMSG, the user's nickname, and so on. So the maximum length of the actual text you're trying to send will vary depending on who you're sending to, the users nickname, your nickname, etc.
Since encoded UTF-8 is longer than plain text, this means that your maximum allowed message length will depend on the final encoding of the UTF-8 text, as well as the above-mentioned issues.
Basically, the UTF-8 will end up scrambled if you're message is too long. As far as I can tell there is no way around that other that other than to set an arbitrarily small limit on the length of your message in the editbox.
Also, mIRC checks a line to see if it contains valid UTF-8 sequences. If it finds anything in the line that is not UTF-8 or invalid UTF-8, it assumes that the line is plain text, or some other type of encoding, and does not perform any UTF-8 parsing on it.
