mIRC Home    About    Download    Register    News    Help

Print Thread
mIRC 6.2 slowdown using $hfind(r) #163659 03/11/06 12:35 AM
Joined: Nov 2006
Posts: 1,559
H
Horstl Offline OP
Hoopy frood
OP Offline
Hoopy frood
H
Joined: Nov 2006
Posts: 1,559
I experienced a vaste slowdown in performance using an addon of mine that had to handle a huge hash'd dataset qick (containing more than 2500 items).
The fallowing alias was made to testify a quite different performance while using $hfind(r) of mIRC 6.2 compared to earlier versions (e.g. 6.16 or 6.17).
I finaly discovered that mIRC 6.2 is a lot more pettish on using wildcards in $hfind(r).
My own workaround was to change expressions and avoid wildcards - if possible -, getting more results first. Then I had to cross-check the results.
This approach takes more time than hfinding "directly" if using mIRC 6.17, but still less if using 6.2.
Ah, don't care about the useless kind of dataset; And I am well aware that THIS kind of checking was useless... It's just for testing purposes smile

Code:
alias hfind.r.test {
  hmake test 10
  hadd test 1 bla!blubb@stuff 123 456 234 b abc!def@ghi
  hadd test 2 bla2!blubb@stuff 123 456 234 b abc!lmn@xyz
  hadd test 3 tublu!abbla@blobb 123 456 234 b bbb!ccc@ddd
  hadd test 4 blblbl!blblbl@boblo 123 456 234 b bu!ba@bil
  hadd test 5 some!other@stuff that should not match at!all@*
  hadd test 6 still!more@other stuff not matching any hfind!r@type

  echo -ts ----- testing mIRC $version started -----
  var %string = b•.*b.*•bl.*!•.*bl.*•b.*l.*!.*@.*•.*b.*l.*!.*@.*•3•.*b.*3.*•.*b.*u.*3•.*b.*u.*3.*b.*, %token = 1
  while ($gettok(%string,%token,149)) {
    var %type = $ifmatch, %loops = 1, %matches = 0, %start = $ticks
    while (%loops < 1000) {
      var %item = 1
      while ($hfind(test,%type,%item,r).data) { inc %matches | inc %item }
      inc %loops
    }
    echo -s Type %token : $calc($ticks - %start) ticks [looking for %type in %loops loops: %matches matches]
    inc %token
  }
  hfree test
  echo -ts ----- testing mIRC $version ended -----
} 


Here the results:

[01:07:52] ----- testing mIRC 6.17 started -----
Type 1 : 454 ticks [looking for b in 1000 loops: 3996 matches]
Type 2 : 453 ticks [looking for .*b.* in 1000 loops: 3996 matches]
Type 3 : 485 ticks [looking for bl.*! in 1000 loops: 3996 matches]
Type 4 : 500 ticks [looking for .*bl.* in 1000 loops: 3996 matches]
Type 5 : 500 ticks [looking for b.*l.*!.*@.* in 1000 loops: 3996 matches]
Type 6 : 547 ticks [looking for .*b.*l.*!.*@.* in 1000 loops: 3996 matches]
Type 7 : 453 ticks [looking for 3 in 1000 loops: 3996 matches]
Type 8 : 500 ticks [looking for .*b.*3.* in 1000 loops: 3996 matches]
Type 9 : 468 ticks [looking for .*b.*u.*3 in 1000 loops: 2997 matches]
Type 10 : 454 ticks [looking for .*b.*u.*3.*b.* in 1000 loops: 2997 matches]
[01:07:57] ----- testing mIRC 6.17 ended -----

You see: no measurable slowdown, whatever ammount of wildcards used.
Now look at mIRC 6.2:

[01:12:21] ----- testing mIRC 6.2 started -----
Type 1 : 422 ticks [looking for b in 1000 loops: 3996 matches]
Type 2 : 515 ticks [looking for .*b.* in 1000 loops: 3996 matches]
Type 3 : 500 ticks [looking for bl.*! in 1000 loops: 3996 matches]
Type 4 : 922 ticks [looking for .*bl.* in 1000 loops: 3996 matches]
Type 5 : 828 ticks [looking for b.*l.*!.*@.* in 1000 loops: 3996 matches]
Type 6 : 2000 ticks [looking for .*b.*l.*!.*@.* in 1000 loops: 3996 matches]
Type 7 : 438 ticks [looking for 3 in 1000 loops: 3996 matches]
Type 8 : 1062 ticks [looking for .*b.*3.* in 1000 loops: 3996 matches]
Type 9 : 1906 ticks [looking for .*b.*u.*3 in 1000 loops: 2997 matches]
Type 10 : 1922 ticks [looking for .*b.*u.*3.*b.* in 1000 loops: 2997 matches]
[01:12:31] ----- testing mIRC 6.2 ended -----

I wondered a lot about this phenomena, and I'm very interested to learn what's the reason for smile Thanks a lot.

Re: mIRC 6.2 slowdown using $hfind(r) #163660 03/11/06 11:45 AM
Joined: Oct 2003
Posts: 313
S
Sais Offline
Fjord artisan
Offline
Fjord artisan
S
Joined: Oct 2003
Posts: 313
Similar results

Code:
----- testing mIRC 6.2 started -----
Type 1 : 360 ticks [looking for b in 1000 loops: 3996 matches]
Type 2 : 422 ticks [looking for .*b.* in 1000 loops: 3996 matches]
Type 3 : 406 ticks [looking for bl.*! in 1000 loops: 3996 matches]
Type 4 : 812 ticks [looking for .*bl.* in 1000 loops: 3996 matches]
Type 5 : 860 ticks [looking for b.*l.*!.*@.* in 1000 loops: 3996 matches]
Type 6 : 1812 ticks [looking for .*b.*l.*!.*@.* in 1000 loops: 3996 matches]
Type 7 : 344 ticks [looking for 3 in 1000 loops: 3996 matches]
Type 8 : 922 ticks [looking for .*b.*3.* in 1000 loops: 3996 matches]
Type 9 : 1719 ticks [looking for .*b.*u.*3 in 1000 loops: 2997 matches]
Type 10 : 1781 ticks [looking for .*b.*u.*3.*b.* in 1000 loops: 2997 matches]
----- testing mIRC 6.2 ended -----


Looks like a change in the PCRE engine, perhaps - the things that take longest are those that start with .* (which makes some kind of sense). ISTR there being a bug fixed in the PCRE library which did make certain searches take longer...but my memory isn't very good smile


Sais
Re: mIRC 6.2 slowdown using $hfind(r) #163661 03/11/06 03:25 PM
Joined: Dec 2002
Posts: 4,563
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,563
The decrease in regex performance is due to a change in PCRE, related to this issue. The latest version of PCRE allows the use of either recursion or heap allocation, and since recursion was resulting in a stack crash, I changed PCRE to use heap allocation. It seems that the heap allocation method seriously affects performance however.

I could switch back to the recursion method and place a limit on recursion depth, which should prevent stack crashes, in order to bring back previous performance levels. I'll see if there's a way to do that safely.

Re: mIRC 6.2 slowdown using $hfind(r) #163662 03/11/06 04:27 PM
Joined: Oct 2003
Posts: 313
S
Sais Offline
Fjord artisan
Offline
Fjord artisan
S
Joined: Oct 2003
Posts: 313
I presume this is a compile-time option. No possibility of adding a run-time option to $reg*? smile


Sais
Re: mIRC 6.2 slowdown using $hfind(r) #163663 03/11/06 11:21 PM
Joined: Dec 2002
Posts: 4,563
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 4,563
It's a compile-time option I'm afraid, one or the other :-)

Re: mIRC 6.2 slowdown using $hfind(r) #163664 03/11/06 11:23 PM
Joined: Apr 2004
Posts: 759
M
Mpdreamz Offline
Hoopy frood
Offline
Hoopy frood
M
Joined: Apr 2004
Posts: 759
The default recursion depth of 10million seems a bit much for a ~940 bytes long string :P


$maybe
Re: mIRC 6.2 slowdown using $hfind(r) #163665 24/11/06 11:55 AM
Joined: Oct 2003
Posts: 313
S
Sais Offline
Fjord artisan
Offline
Fjord artisan
S
Joined: Oct 2003
Posts: 313
Looking better in 6.21 smile
Code:
----- testing mIRC 6.21 started -----
Type 1 : 343 ticks [looking for b in 1000 loops: 3996 matches]
Type 2 : 344 ticks [looking for .*b.* in 1000 loops: 3996 matches]
Type 3 : 328 ticks [looking for bl.*! in 1000 loops: 3996 matches]
Type 4 : 375 ticks [looking for .*bl.* in 1000 loops: 3996 matches]
Type 5 : 359 ticks [looking for b.*l.*!.*@.* in 1000 loops: 3996 matches]
Type 6 : 453 ticks [looking for .*b.*l.*!.*@.* in 1000 loops: 3996 matches]
Type 7 : 344 ticks [looking for 3 in 1000 loops: 3996 matches]
Type 8 : 375 ticks [looking for .*b.*3.* in 1000 loops: 3996 matches]
Type 9 : 360 ticks [looking for .*b.*u.*3 in 1000 loops: 2997 matches]
Type 10 : 515 ticks [looking for .*b.*u.*3.*b.* in 1000 loops: 2997 matches]
----- testing mIRC 6.21 ended -----


(Just found the alias kicking about while cleaning up, so tried it out)


Sais