UTF-8 display bug - mIRC Discussion Forums

Hello, I want to report a bug for Moo\ who has problems with receiving the registration mail.

Quote:

Sorry if this has already been reported..
I did a quick search for "utf8" and the topics seemed to discuss different issues - so...

The problem is about UTF8-Display together with bold or underlined text.
If both appear in one line, the performance of mirc will be decreased heavily, including scrolling, copying text, simply everything concerning the window while the affected text is inside it.
Scrolling away or switching the window results in mirc working normally again, it's just about the time those lines are visible.

To reproduce it, you could simply run the following commands: (utf8-display should be on)

Code:

//var %x = 1 | while (%x <= 100) { echo -ag test %x - äöü äöü äöü | inc %x } | while (%x <= 200) { echo -ag test %x - äöü äöü äöü | inc %x }

As you should have noticed, the first 100 lines are displayed normally, the other 100 decrease the display-speed continuously.
Try scrolling between those 2 blocks of text (especially using the mouse wheel), copy some lines, you will notice the difference quickly.

I for myself can reproduce it with mIRC 6.32

thanks

I see no problems with that code here. Runs fast and has no display slowdowns.

What font are you using?

Here we go...
After a 3rd try, using an email hoster different to gmx.de this time, the email came immediately. (I am Moo\ :P)

About the font, I'm using Verdana, font style standard, size 8.
Make sure you got your UTF8-Display turned on in mIRC options..
I'm using the UTF-8 setting "default" in the font dialog by the way...
mIRC Version 6.32.

Okay I was able to notice a *tiny* slowdown- it's due to Font Linking. Disable font-linking and you should be fine: alt-o -> irc -> messages -> [ ] font linking

This is not really a bug, just a reality of how Font Linking works: it's bound to be slower. I imagine it's just being accentuated by the use of control codes in this example. If your font is unicode aware (like verdana is) then you don't need to enable font-linking.

edit:

By the way, I ran the following benchmarks based on your code:

Code:

alias bench_normal {
  window @win
  var %x = 1 
  while (%x <= $$1) { 
    echo @win test %x - $2-
    inc %x 
  } 
}

alias bench_bold {
  window @win
  var %x = 1
  while (%x <= $$1) { 
    echo @win test %x - $2-
    inc %x 
  }
}

I use a /bench utility that runs them each and records ticks, I get:

1. With no unicode:

Command completed in (avg over 100 runs) 50.7 ticks: bench_normal 100 hello world
Command completed in (avg over 100 runs) 102.96 ticks: bench_bold 100 hello world

So bold is normally twice as slow, but we knew that.

2. With unicode, linking off:

Command completed in (avg over 100 runs) 54.75 ticks: bench_normal 100 äöü äöü äöü
Command completed in (avg over 100 runs) 96.72 ticks: bench_bold 100 äöü äöü äöü

3. With unicode, font linking on:

Command completed in (avg over 100 runs) 63.49 ticks: bench_normal 100 äöü äöü äöü
Command completed in (avg over 100 runs) 95.32 ticks: bench_bold 100 äöü äöü äöü

So empirical evidence shows there's actually *no* slowdown, even though the display might look slightly more lagged visually.

It looks like Font Linking is really the problem in this case...
But to be honest, I do not consider this a "tiny" slowdown, it's pretty hard imo.
I have saved the current $ticks value 3 times, before the displaying of the text, after the first 100 lines and after all 200 lines.
Then I was able to calculate the relative time needed to display those lines.
While the first 100 lines needed 125 ticks, it took 4031 to display the bold-utf8-lines.
Also, many other processes (as I already mentioned copying and scrolling as most important ones) are slowed down. (not just slightly, but strongly delayed)

I've asked 3 other mirc-users if they can reproduce this slowdown before I have posted here. (one of them has been tonyp)
Everyone affirmed my observance, after re-asking one of them, I was told about 5-7 seconds.

Anyway, I'm going to disable font linking so far since I do not think it had any use for me at all.

I am also able to reproduce this issue, however it depends on the font and script that I am using. Unfortunately I don't know of a way to improve the speed in the current implementation. Font linking is an intensive process and requires mIRC to scan all existing fonts on your system for any matching scripts or characters for every line and character that is displayed. Since mIRC is a non-unicode application, it has to perform this process manually. In a future unicode version of mIRC, this will not be necessary and should result in a faster display.

Can't you create something like a search index on startup?
Or do this by a slower, continuous process after starting?

Also, I do not know if I understood the function of font linking correctly, but as fast as I could get it, its made to display characters which are not provided by the current font.
So, there is actually no need to scan all of my fonts, since I was not using any non-Verdana characters...
Doesn't mIRC first look up the char in the currently used font before searching for it elsewhere?

Sorry if I misunderstood the problem :s

it still has to do the check though, even if it can use the default font.

Yes that's why I originally said "you probably don't need font-linking anyway".. you still might, since verdana doesn't have *every* unicode codepoint, but its less likely with a proper unicode font.

I'm not sure why it does the lookup even when it decides to use the verdana glyphs though.. this might be necessary, but it might just be a flaw in mIRC's implementation.

mIRC calls the Windows font-linking API and uses the result that Windows provides. Windows itself performs the scanning and font-matching, which tries to find the required characters in the current font before searching other fonts. mIRC enhances this by using its own strategies to speed up the process and internal caching for the most recently used fonts to avoid repeated scans. It's quite a complicated process. In some cases, and for some fonts and characters, repeated searches are necessary, due to the way that Windows works. It also may work differently under different versions of Windows and may match fonts differently depending on how your system is configured. I spent many months working on it when I first implemented it. As far as I can see, it is as fast as it can be without converting mIRC to a unicode application.

Okay, if there's no easy solution to improve it, I will have to wait for a proper unicode release

However, I can't get why this only appears using bold or underlined text, but that's not surprising as I do not know the way mIRC looks up the characters...

Thank you for dealing with the issue anyway

I can't think of a reason why it would show such a significant slow-down for you when you use a bold or underline character. When I use the same font and script that you mention in your post, I experience a small slow-down here, which is expected, both when scrolling and marking/copying text. However it's nowhere near several seconds. Which version of Windows are you using? Is it the English language version? What kind of CPU do you have?

I think, the reason for my extreme slow-down is that I am using "extreme" settings.
I mean, my resolution is 1680x1050, mIRC and the channel window in it are maximized, my font size is 8...
The more characters are visible on the page, the slower the computer reacts. (64 lines on the page here)
Anyway, I've also tested characters which actually require font-linking to be enabled, that has been even worse.

Code:

alias utfcheck {
  if (!$3) { echo -ag /utfcheck <1|2> <characters> <lines> | return }
  var %x = 0, %y = $replace($1,1,à¹‘Û©ÛžÛ©à¹‘,2,10à¹‘14Û©ÛžÛ©10à¹‘), %z = $str(%y $+ $chr(32),$2), %ticks = $ticks
  while (%x < $3) {
    inc %x
    echo -ag %x %z
  }
  echo -ag finished (type $1 $+ ); %x lines displayed in $calc($ticks - %ticks) ticks;
}

Quote:

/utfcheck 1 3 50
finished (type 1); 50 lines displayed in 141 ticks;

/utfcheck 2 3 50
finished (type 2); 50 lines displayed in 6578 ticks;

I'm using a German version of Windows XP, Service Pack 3.
My CPU won't be the problem.. it's a quad core.
"Intel® Core™2 Quad CPU Q6600"
Running at 4x2.75ghz at the moment...

Also, my GPU is not the oldest, NVIDIA GeForce 8800 GTX.

I have just tried setting ClearType on (in Windows settings), which only made a small difference.. it's a bit slower now.

By the way, I was using Service Pack 2 before...
Tried updating, but that did not really help.

Here again my font settings:
http://img234.imageshack.us/img234/5628/unbenanntiq4.jpg
http://img234.imageshack.us/img234/2705/unbenanntsx7.jpg

Edit:
I have asked some other people to test this...
Here the results: (is being updated)

Quote:

My other computer:
finished (type 1); 50 lines displayed in 172 ticks;
finished (type 2); 50 lines displayed in 5203 ticks;

Some other guys:
finished (type 1); 50 lines displayed in 281 ticks;
finished (type 2); 50 lines displayed in 3016 ticks;

finished (type 1); 50 lines displayed in 391 ticks;
finished (type 2); 50 lines displayed in 3922 ticks;

finished (type 1); 50 lines displayed in 93 ticks;
finished (type 2); 50 lines displayed in 2782 ticks;

finished (type 1); 50 lines displayed in 250 ticks;
finished (type 2); 50 lines displayed in 3516 ticks;

My system and graphics card are much less powerful than yours. If I change my display settings to 1280x1024, maximize mIRC and then run your script, my results are:

Quote:

Cleartype off:
finished (type 1); 50 lines displayed in 297 ticks;
finished (type 2); 50 lines displayed in 1891 ticks;

Cleartype on:
finished (type 1); 50 lines displayed in 906 ticks;
finished (type 2); 50 lines displayed in 3672 ticks;

It's difficult to know exactly why some users experience a greater slow-down. However, I'm not too surprised considering how complex the font-linking feature is. It really depends on so many different issues. I think we will just have to wait until mIRC becomes a full unicode application to resolve this.

I guess so, too...
And when is this expected to happen, if I may ask?

(It's not about time... just if you're planning to implement it in the next versions.)

Btw, i have a question for myself and I hope I'll get an answer here:

Why re $utfencode, $utfdecode and $isutf undocumented?
They are really usefull in my eyes...

Thanks, they are listed in the versions.txt file, however it looks like I forgot to add them to the help file. I'll add them for the next version.

The short answer: soon :-) The long answer: turning mIRC into a full unicode application will require a huge amount of work and time. I plan to start the unicode conversion some time in the next month or two. Once I start, it will be very difficult to add new features to mIRC, or to make any other changes, so I plan to release one or two bug fixes beforehand to ensure that the mIRC 6.3x series is stable and has no major issues. I have no idea how long it will take to convert mIRC to unicode. There are 150000+ lines of code. I will most likely need to release a public beta at some point since it will need some serious, wide-spread testing before a stable release is ready.