Forums Bug Reports Unicode input problem

Hi there

I'm writing to report a problem with inputting complex Unicode in mIRC. I'm using mIRC 6.21 on XP SP2 and the problem has been reproduced by a number of other people.

mIRC of course now supports Unicode using UTF-8 and I am very thankful for that great improvement, but there is a persistent problem with inputting some Unicode characters.

Specifically, I have no problem typing in, say, Greek, but whenever I try to input polytonic Greek, the mIRC edit box will just give me question marks (replacement characters) instead. Similarly, when I press enter, these will be transmitted as question marks. The really odd thing is that this problem only appears when I input the text directly into mIRC itself - but is wholly absent when I first enter the text into a standard windows app (like Notepad) and then subsequently paste it into mIRC. In the latter case, the characters show up fine both in the edit box and in the channel window. This goes for a number of "complex" Unicode glyphs like polytonic Greek and Devanagari (Indian) - and I think I saw another poster mentioning the same problem with Korean. I am not forgetting to switch on "Display and encode" and it cannot be a font problem, as there is no problem copying and pasting the very same text that will not input directly into mIRC.

This problem is not unique to mIRC, tho. I've noticed it in at least two other programs, both rather small ones. The first is KeyNote by Trianglos Software, a great (but discontinued) note-taking program, and the second is Gloz, a small program made by a friend of mine to aid memorization tasks. Both of these display the exact same problem as the one described above. My earlier enquiries into the problem suggest that it may have something to do with the RichEdit-module that is apparently used by all these programs to handle input, but I am no programmer and cannot say for sure.

Now, the problem I've described above seems to be connected with two other problems, both of which also appear in the two other programs I've mentioned. The first concerns the Windows language bar (located in the Windows taskbar). This language bar allows you to switch between different languages, and there's a Windows option that allows you to do this using a hotkey combination. I myself use five different language layouts and switch between them using the combination CTRL+SHIFT which will step through the cycle JP > DA > EL > HE > SA and then back to JP. This usually works as a charm, with the exception of the three above-mentioned programs. In, say, mIRC when I press CTRL+SHIFT it'll switch from DA to EL, then to HE, but then go back to EL again, skipping SA, JP and DA. Then it'll just cycle between HE and EL and if I want to go any other language, I need to do it manually, using the mouse to click said language. Very weird.

What's perhaps even weirder is use of font substitution for apparently no reason at all in some cases. For instance, the word "Devanagari" is properly written with a macron (horizontal bar) above the second a and the final i (can't reproduce it here on the board). When I paste this into mIRC (or indeed any of those programs), the a-macron and the i-macron will be rendered by font substitution even though the font used supports both a-macron and i-macron. Very, very weird.

But it gets weirder still. All these bugs appear when I try to enter stuff into an mIRC channel edit box or into the Scripts editor, but they do not appear when inputting text into other mIRC edit boxes, like the Address Book or the many boxes within the mIRC Options dialogue. I'm assuming this is due to these boxes being handled by some other Windows component than the one that handles the channel and script editor edit boxes.

Anyway, this should serve as a description of the bug. As I said, it's been reproduced by others and it may well be related to the RichEdit module. Could it be solved by merely updating that - or is the problem more complex? Hope to get some feedback on this.

/Rasmus

I've seen this as well and while in some ways it is just strange, it is also pretty serious as it prevents typing many characters/languages at all. I have gotten used to the type-in-notepad, paste-in-mirc workaround, but it is tedious. Bwuser says he has seen it in other software; I haven't though, mIRC is the only thing I have with the input errors.

For an idea of how very random a lot of it seems:

I pasted in mIRC all the Latin letters in the Latin Extended-A range, which contains a lot of the letters needed for major European languages. The great majority of the letters do not get substituted, but I can TYPE none of them. The ones that do get substituted are all the lowercase letters with macrons, lowercase e with caron, lowercase n with acute accent and lowercase n with caron. The capital forms of all these letters do not get substituted; I have tested this with several fonts, all of which have the required pre-composed glyphs, so ... huh?

So really it's two problems, maybe separate:
1. mIRC's editbox doesn't let you type in a lot of Unicode characters, but does allow you to paste them if you type them in notepad, etc. first.
2. mIRC's editbox substitutes certain, specific characters even when it is not needed. notably, this only happens in the editbox .. once you press enter, the text shows up correctly in channel in the same font.

I confirm the same problem here.
It seems mIRC will transform most of my Latin characters if I use a Japanese locale, but only on keyboard input, copy/paste works fine.

That means à becomes a, é/è becomes e, and all of that regardless of messages option in mIRC, or of the input method, which, by the way, work perfectly fine in exlorer/ie/firefox/notepad/emeditor and most applications I have been using(ie, typing the key for é prints e, but copy/pasting é in the editbox works perfectly fine).

I think I reported this in the past, but without success.(that is no answers)

On a side, if that helps in anyway:
If multi-byte edit box is enabled, typing non-localized characters results in garbled output.

Problem confirmed as still being in 6.3.
(I thought so when I did not see it in the whatsnew.txt)

Typing using any cjk(chinese/japanese/korean) input method does not let the accentuated chars(é/è/à etc) go through, but converts to their unaccentuated counterparts(e/e/a etc).
The problem does not appear in almost all other applications I tried(internet explorer, firefox, skype, google talk, photoshop, notepad, etc).(I had the same problem in skype though)

It's interesting that I can still enter those using the multibyte editbox by using shortcuts on a US layout...

Ctrl-' then e shows é
Ctrl-` then e shows è
Ctrl-^ (ctrl-shift-6) then o shows ô
Ctrl-: (ctrl-shift-;) then o shows ö
Ctrl-~ (ctrl-shift-`) then n shows ñ

...and so on.

I wonder why that's not affected. I'm not surprised that Alt codes work fine as those don't need multibyte display to function, but the shortcuts I mentioned *do* need multibyte display to work, so I'd expect them to be similar to the other languages, but I guess not.

It's very likely that this is due to the RichEdit editbox, since mIRC has very little control over the way the RichEdit editbox displays text. mIRC uses an older version of the RichEdit editbox (riched20.dll) which is consistent across all versions of Windows. However this means that it may have less functionality than the newer versions of RichEdit. I will make a change in the next version of mIRC so that it loads the newer version (msftedit.dll) under XP and Vista - it should have better support for displaying a wider range of fonts.