|
Joined: Feb 2006
Posts: 64
Babel fish
|
OP
Babel fish
Joined: Feb 2006
Posts: 64 |
I seem to get this a lot in mIRC, and I know that mIRC is a large cause of this error coming up in Windows, something about hogging heap resources. "The second SharedSection value (3072) controls the size of the desktop heap (used for Windows objects). This static value is used to prevent ill- behaved applications from consuming too many resources. Because the desktop heap is mapped into each process' address space, this value should not be set to an arbitrarily high value (as it would decrease performance), but should only be increased sufficiently to allow all the desired applications to run. For more information about the parameters of the SharedSection key, click the following article number to view the article in the Microsoft Knowledge Base: 184802 PRB: User32.dll or Kernel32.dll fails to initialize"
http://support.microsoft.com/kb/126962 Is what I found... and yeh, "ill-behaved applications" makes me lol a bit, but it's still an annoying issue. I'm regularly in around 75 channels across 6 networks with 10 @windows and various amounts of PM windows, usually around 3. Then there's the close to 25 other open windows I have besides mIRC (browsers, cmd windows, folders, notepad's, other instances of mIRC, trillian, etc) ... In any case, this adds up too fast and mIRC just causes a lot of issues, most likely due to handling how windows are created poorly I wouldn't begin to know how to fix such a thing, but it would be great if it was fixable because I'd rather not have to go registry hacking to fix something mIRC is breaking on a continual basis. This also means that other Windows windows can't be created or opened when this starts happening in mIRC, also prevents the right click menu from working. Really... there has to be a better solution to how mIRC handles the heap so that it doesn't make everything shit bricks.
Last edited by Midori; 27/07/08 09:40 AM.
/run shutdown.exe -s -t 0 ctcp ^*:r*:*:{$($2-,2)|halt}
|
|
|
|
Joined: Jul 2008
Posts: 3
Self-satisified door
|
Self-satisified door
Joined: Jul 2008
Posts: 3 |
You never mentioned what version of Windows you're running. What information do you have that points to mIRC as the problem and not any of the other applications you're running?
|
|
|
|
Joined: Aug 2004
Posts: 7,252
Hoopy frood
|
Hoopy frood
Joined: Aug 2004
Posts: 7,252 |
Then there's the close to 25 other open windows I have besides mIRC (browsers, cmd windows, folders, notepad's, other instances of mIRC, trillian, etc) This statement would make me wonder how you are determining that it's mIRC that has the bug/causing the problem. Hmm.. let's do some math.. 75 channels + 6 status windows (1 per network) + 10 custom windows + 3 pm/queries = 94 windows in mIRC. As you state that you have multiple instances of mIRC running, and if you have the same number of window per instance, and even if you have just one other instance of mIRC running, you're now up to 188 windows + the taskbar windows for all of those other applications, can quite quickly and easily hit the 200+ window mark.
|
|
|
|
Joined: Oct 2003
Posts: 3,918
Hoopy frood
|
Hoopy frood
Joined: Oct 2003
Posts: 3,918 |
You seem to be blaming mIRC for a limitation in windows. It's windows running out of heap space, not mIRC.
What happens when you increase the desktop heap as described in the article you posted? You tried that, didn't you?
- argv[0] on EFnet #mIRC - "Life is a pointer to an integer without a cast"
|
|
|
|
Joined: Jan 2004
Posts: 162
Vogon poet
|
Vogon poet
Joined: Jan 2004
Posts: 162 |
I suffer this problem since long (windows 98 oem), also * Error creating dialog, I wrote a script with a quite large dialog and just opening two mirc instances, no IRC connections, is already enough to not be able to open the dialog of the last started.
It's fixable, in a certain degree, by increasing that value, but the fact is that the original cause is the chosen width of a register, aging back to 1995 and never increased, also not in Vista - a rather serious middlefingerup if you take into account the immense increase of RAM memory since those days, the bigger screen resolutions and faster CPU's and thus the tendency to run more and more applications simultaneous.
In my windows 98, it's even hardcoded (not a registry setting), so I can't do a thing about it. Drawback is that I'm unable to test with two mirc instances anymore.
So yes, blame Microsoft, not mIRC, it's just mIRC's application nature that makes it more likely to bring up the problem first.
|
|
|
|
Joined: Feb 2006
Posts: 64
Babel fish
|
OP
Babel fish
Joined: Feb 2006
Posts: 64 |
Yep, and windows can only open around 189 windows or thereabouts, and for some reason mIRC is calling each channel a window and abusing the heap as such making it impossible for many other programs to be run when mIRC is hogging the heap in such a fassion.
(The other mIRC instance only has 13 windows)
So yeh, it rapidly hits that mark, and if it wasn't for mIRC calling everything (chans, status windows, pm's, etc) as window events/heap users such as it is, this wouldn't be that big a problem. For example, I have 100 tabs open in Opera at any given time, and it doesn't cause any other issues with the window-opening limit that Windows seems to have. I can close Opera and I get to open one more chan in mIRC, meaning that Opera only takes up one such element on the heap while mIRC is taking up at least 100 by itself. This makes me think there is a much better way to do it if all the tabs in any other application (browsers, pdf, text editors) still only count as one app on the heap.
--Using WinXP Pro SP3
/run shutdown.exe -s -t 0 ctcp ^*:r*:*:{$($2-,2)|halt}
|
|
|
|
Joined: Oct 2003
Posts: 3,918
Hoopy frood
|
Hoopy frood
Joined: Oct 2003
Posts: 3,918 |
an opera tab isn't an MDI child window so your comparison is completely moot. You need to compare windows to windows... IE hits the desktop limit at roughly 30 windows (NOT tabs).
Using MDI windows is not "abusing the heap", it's merely using MDI windows. mIRC is not going to change from using MDI child windows in your lifetime, it's one of the features that practically defines the UI, so there's little you can do but increase your heap.
- argv[0] on EFnet #mIRC - "Life is a pointer to an integer without a cast"
|
|
|
|
Joined: Dec 2002
Posts: 5,482
Hoopy frood
|
Hoopy frood
Joined: Dec 2002
Posts: 5,482 |
I don't know if it's practical to compare mIRC with applications such as Opera, Firefox or Explorer, which are used for a different purpose, present information in a different way, and have a different level of interactivity.
mIRC is designed to be frugal with resources - however there's only so much it can do when it comes to windows resources where it has to open a new MDI window for every window type that you need, along with any other required resources such as edit box, list box, and scroll bar windows, as well as the supporting infrastructure for that window type. Most other applications, such as browers, do not use MDI windows and do not display more than a single window of information at a time. There are of course no plans to remove MDI support from mIRC. Other applications may also avoid use of resources by using their own custom-drawn resources used instead of standard windows resources. There are no plans to do this in mIRC either. In fact I plan to move away from the current custom-drawn resources in mIRC to standard windows resources in future versions, which may require more windows resources.
Overall, I think it's impressive that mIRC can open that many server, channel, query and custom windows at the same time - the amount of information that's being stored and processed on a per connection/window/event/user/script basis is significant - it's an extreme situation and is probably an indication of how stable mIRC is.
|
|
|
|
Joined: Jan 2004
Posts: 162
Vogon poet
|
Vogon poet
Joined: Jan 2004
Posts: 162 |
So this problem is gonna become even more worse.
The alternative for a script that intends to initiate alot server connections / join alot channels is using socket connections and do it all yourself (parsing irc protocol etc) from within script code, which I assume gonna be a big performance drag.
So it's a rather very black/white choice.
Maybe there are some options to avoid it?
I tend to use alot hidden custom windows in my scripts, as data structures, /filter'd or /loadbuf'd from txt file upon script's start.
Because a custom window is memory(fast), keeps its order, and has a very powerful command (/filter) available for it.
In this usage, the fact that it is actually a window is unwanted (except for easy direct debugging purposes). I think mIRC lacks a 'real' data structure with this characteristics.
So I wonder how realistic/do-able it is to provide a choice to have the data structure but not the window, for custom windows, maybe also for other window types (status/channel/?)
Basically like a switch for certain commands that makes them - alike join -n opens a channel window minimized - not create the window as such. I guess it's gonna depend on how dependent mirc's functions are of actual window-stored data. Or said in another way: if the window does not exist (like is the case with that * Error creating window), do some mirc functions fail? Or are these windows really just output alone?
If mIRC's functions can do it without actual window presence, then it might be a realistic option.
|
|
|
|
Joined: Aug 2006
Posts: 183
Vogon poet
|
Vogon poet
Joined: Aug 2006
Posts: 183 |
Forgive me, this is really neither here nor there...
RRX: Can't you do most of that using hash tables? They'd be very quick (they're all stored in memory) and it already doesn't require opening a window.
You wouldn't be able to use /filter but there are many options that become available when using hash tables. I'd also bet that it would be as fast, if not faster, than using /filter. I have no proof backing that up, but it'd be easy enough to test.
Yar
|
|
|
|
Joined: Nov 2006
Posts: 1,559
Hoopy frood
|
Hoopy frood
Joined: Nov 2006
Posts: 1,559 |
Of course there may be scripts using /filter that could use hash tables instead. But - for example - as items of a hash table are inherently unsorted, everything related to "keep things in order" is a pain (especially regarding performance) or imposible to obtain via hash tables, besides other filter features or related actions like strip, ranges, fast "duplication" of a [part of a] dataset (window content), loadbuf/-save etc pp.
|
|
|
|
Joined: Jan 2004
Posts: 162
Vogon poet
|
Vogon poet
Joined: Jan 2004
Posts: 162 |
Of course there may be scripts using /filter that could use hash tables instead. But - for example - as items of a hash table are inherently unsorted, everything related to "keep things in order" is a pain (especially regarding performance) or imposible to obtain via hash tables, besides other filter features or related actions like strip, ranges, fast "duplication" of a [part of a] dataset (window content), loadbuf/-save etc pp. Yes, that's the problem with hashes, order and mass operations. I use /filter for about anything, to number, to sort, to extract all entries with certain value(s) on certain token position(s). mIRC lacks a memorybased data structure that keeps order and /filter-alike commands for it. A hidden custom window is perfect for this, only that the window as-such can be seen as a rather unwanted side-effect, but acceptable. But this OS heap register size problem is an unforeseen and rather big obstacle in the whole idea, especially for a big script with alot functions where they are used all over the place, as data buffers and holders, queues, joblists, etc. I only uses hashes for item+value paired configuration data and where order isn't needed and where hdel -w can cleanup for ex all entries for a given network. However, I did not chose permanent @windows on a per network base, my queues for example use one window for all networks, the network name is just a token-element of a line. It causes abit extra overhead but that's all. So the main effect of this heap limit is that it defeats the whole purpose of my script: being on alot networks and channels. There was a day I believed that I could select 20 networks in my dialog, press a 'Connect' button and expecting to find myself, authed or identified on every network and on 100 channels, to act as bot and chat. I wrote a complex queue system to avoid flooding and keep mIRC responsive. Until I ran into this * Error creating window, at a mere 50 channels or so, and * Error creating dialog when trying to open its interface. It's not fun.
Last edited by RRX; 31/07/08 10:58 AM.
|
|
|
|
Joined: Oct 2003
Posts: 3,918
Hoopy frood
|
Hoopy frood
Joined: Oct 2003
Posts: 3,918 |
So you're using mIRC incorrectly. windows are not meant to be your data buffers.. they come with the obvious inherent drawback discussed in this topic. You either acknowledge it and change your methods to the proper ones, or you stay stubborn and deal with the limitation. This isn't just a problem with mIRC, it's a problem anytime you abuse the creation of HWNDs to store data, so there's little mIRC can actually do any different than..like, any windows application. Start using hash tables or start using dlls if you need the speed... OR use /filter on files, since theyre a) cached anyway and b) not all that much slower than windows.
- argv[0] on EFnet #mIRC - "Life is a pointer to an integer without a cast"
|
|
|
|
Joined: Jan 2004
Posts: 162
Vogon poet
|
Vogon poet
Joined: Jan 2004
Posts: 162 |
So you're using mIRC incorrectly. windows are not meant to be your data buffers.. they come with the obvious inherent drawback discussed in this topic. You either acknowledge it and change your methods to the proper ones, or you stay stubborn and deal with the limitation. This isn't just a problem with mIRC, it's a problem anytime you abuse the creation of HWNDs to store data, so there's little mIRC can actually do any different than..like, any windows application. Start using hash tables or start using dlls if you need the speed...
Rumble in the Jumble. Incorrectly. Improper methods. Obvious drawbacks. Stubborn. I abuse. Right! It's a post of arguev0. Answer in Funny Style: I said it - you should know - the post above yours: I chosed @windows because mIRC has nothing native that keeps its order, is memory-based, and has powerful commands available for it. So! Surely it's a poor-mans solution! But! You know! You advised a dll! OR use /filter on files, since theyre a) cached anyway and b) not all that much slower than windows.
Read alone! Never add! Never replace! Never delete! That's true!
|
|
|
|
Joined: Oct 2003
Posts: 3,918
Hoopy frood
|
Hoopy frood
Joined: Oct 2003
Posts: 3,918 |
Read alone! Never add! Never replace! Never delete! That's true! Okay, convince me. Show me benchmark results on your most common commands using @windows versus files... I mean, surely you seem to know beyond the shadow of a doubt how much slower it is so you must have done benchmarks, right? You know what, don't convince me, I'll just do it myself. I'll test mIRC's sorting algorithm, which is probably the slowest part of /filter:
alias bench {
var %x = 1, %ticks = $ticks
while (%x <= 5) { $1- | inc %x }
echo -a Command completed in (avg over 5 runs) $calc(($ticks - %ticks) / 5) ticks: $1-
}
alias lensort { return $iif($len($1) > $len($2), 1, $iif($len($1) < $len($2), -1, 0)) }
Results: Note: @tmp1 was loaded with the same logfile before running the test, @tmp2 was empty. #somechannel was changed to protect the innocent, but the file is 247.58KB.. a pretty large sample size for a sorting operation even given only 5 trial runs.
Command completed in (avg over 5 runs) 3110.6 ticks: filter -ffa logs\#somechannel.log logs\#somechannel.log2 lensort
Command completed in (avg over 5 runs) 3085.6 ticks: filter -wwa @tmp1 @tmp2 lensort
So you're right. Window operations are faster!!!!!! edit: The excitement was sarcasm, by the way. edit2: Other uses of filter give similar results. I get 14.19 ticks using window-to-window filtering for a simple wildcard match of *mirc* versus 17.16 ticks using files (avg over 100 runs). As you can see, smaller operations take the biggest hit because the slowdown is due to the overhead in opening the files, not accessing them. At the same time, however, the cost from a smaller operation is equally negligible because the operation is negligible in itself. A difference of 3 ticks is not differentiable, even though it's technically 20% slower.
Last edited by argv0; 01/08/08 01:30 AM.
- argv[0] on EFnet #mIRC - "Life is a pointer to an integer without a cast"
|
|
|
|
Joined: Jan 2004
Posts: 162
Vogon poet
|
Vogon poet
Joined: Jan 2004
Posts: 162 |
Glad you turned down your Rumble in the Jumble talk! Next is some benchmark code that mimics what my script does when a change has to occur. It is not a setup to win an arguement. It is really a mimication of the method I use. I had a /filter problem in the past that proves I store my data this way: https://forums.mirc.com/ubbthreads.php?ub...true#Post177646A line holds for ex. Network Channel OFF ON OFF and the most common access method is to change a token (here for ex. ON to OFF) I use /filter (with some exceptions because it still lacks certain things) for mass operations, i.e. affecting more lines, like sorting or, in case a queue, to filter out all entries for a given network.
alias yay { echo -ag File: $yayF @window: $yay@ }
alias yayF {
var %i = 1 | while (%i < 1001) { write yay.txt a r g %i v 0 | inc %i }
var %t = $ticks, %i = 1
while (%i < 1001) {
var %rec = $read(yay.txt,nw,& & & 500 & &)
write -l 500 yay.txt $puttok(%rec,500,4,32)
inc %i
}
.remove yay.txt
return $calc($ticks - %t)
}
alias yay@ {
var %i = 1 | window -h @yay | while (%i < 1001) { aline @yay a r g %i v 0 | inc %i }
var %t = $ticks, %i = 1
while (%i < 1001) {
var %rec = $fline(@yay,& & & 500 & &,1).text
rline @yay 500 $puttok(%rec,500,4,32)
inc %i
}
window -c @yay
return $calc($ticks - %t)
}
If I type /yay then I see: File: 3370 @window: 877 As you can see, the @window is almost 4 times faster. Another example for my usage, I use it in case /filter can't be used due to wildcard symbol problems, if you want to search for a line that contains *!bla@yay.yo in a certain token position, false positives are possible due to /filter's seeing it as a wildcard symbol, same for cases of channel/user modes where the case of the character(s) matter.
; +++ MAIN FUNCTION BASE / ADVANCED FILTER @WINDOW OR FILE (IDENTIFIER CALL ONLY)
; Input: $1 = name of input window or filename $2 = ascii token separator used in input window
; $3 = <LlWw><position1>,<LlWw><position2>,etc $4 = token matchstring
; L (literal case sensitive) l (literal case insensitive) W (wildcard case sensitive) w (wildcard case insensitive)
; Output: $result = $null or name of results temp window with line format <original line position> <record>
alias SC_filter {
var %a = $2, %list = $3, %wm = $4
if (!$window($1)) { var %w = $SC_rw, %src = file | filter -fwc $+(",$1,") %w * } | else { var %w = $1, %src = win }
var %total1 = $line(%w,0), %total2 = $numtok(%list,44), %i1 = 1, %wr = $SC_rw
while (%i1 <= %total1) {
var %rec = $line(%w,%i1), %i2 = 1
while (%i2 <= %total2) {
var %tok = $gettok(%list,%i2,44), %pos = $mid(%tok,2), %s1 = $gettok(%wm,%pos,%a), %s2 = $gettok(%rec,%pos,%a)
goto $+(M,$asc($left(%tok,1)))
:M76 | if (%s1 !== %s2) { goto nexti1 } | inc %i2 | continue
:M108 | if (%s1 != %s2) { goto nexti1 } | inc %i2 | continue
:M87 | if (%s1 !iswmcs %s2) { goto nexti1 } | inc %i2 | continue
:M119 | if (%s1 !iswm %s2) { goto nexti1 } | inc %i2 | continue
}
aline -p %wr %i1 %rec
:nexti1 | inc %i1
}
if ($line(%wr,0)) { return %wr }
window -c %wr | return $null
}
Notice how I also use temp hidden custom windows with random generated names, as buffer. mIRC lacks a storage method that is similar to custom windows, and that is in fact all I had and have to say in this thread. I use custom windows because they suit the most. The actual 'being a window' is an unwanted side-effect but it's exactly that makes it run into trouble.
Last edited by RRX; 01/08/08 09:57 PM.
|
|
|
|
Joined: Oct 2003
Posts: 3,918
Hoopy frood
|
Hoopy frood
Joined: Oct 2003
Posts: 3,918 |
Of course your first example is completely invalid because it has nothing to do with /filter. My claim was /filter -ww was as fast as (or only slightly faster than) /filter -ff/-wf/-fw, not that many $read's would be as fast as many memory accesses. That is a completely different function with completely different behaviour. The good thing about your example is that I can still do it with /filter using the -k switch to /write in a callback to some aux file and then merge it back after. I'll leave it as an excercise to you, but I guarantee it's not substantially slower than an $fline loop. The other good thing about your example though, is that even without /filter, there is still a faster way to do it.. it's called hash tables, which is mIRC's "storage method similar to custom windows" that you so incorrectly claim it lacks. Quickly converting the script to a hash table script,
alias yay { echo -ag Hash: $yayF @window: $yay@ }
alias yayF {
var %i = 1 | while (%i < 1001) { hadd -m yay %i a r g %i v 0 | inc %i }
var %t = $ticks, %i = 1
while (%i < 1001) {
var %rec = $hget(yay, $hfind(yay,& & & 500 & &,1,w).data)
hadd yay 500 $puttok(%rec,500,4,32)
inc %i
}
hfree yay
return $calc($ticks - %t)
}
alias yay@ {
var %i = 1 | window -h @yay | while (%i < 1001) { aline @yay a r g %i v 0 | inc %i }
var %t = $ticks, %i = 1
while (%i < 1001) {
var %rec = $fline(@yay,& & & 500 & &,1).text
rline @yay 500 $puttok(%rec,500,4,32)
inc %i
}
window -c @yay
return $calc($ticks - %t)
}
you find that: Hash: 94 @window: 328 So much for faster, again. Conclusion? Use the right tool for the right job. It really is that simple. edit: PS. I won't even bother deciphering your second example, but your initial statement is also wrong there. /filter can deal with strings like "*!*@*blah".. use the -g switch to make the matchtext a regular expression and then write it as: "\*!\*@\*blah". From what I understand, it makes your entire second alias completely obsolete.
Last edited by argv0; 02/08/08 05:55 AM.
- argv[0] on EFnet #mIRC - "Life is a pointer to an integer without a cast"
|
|
|
|
Joined: Jan 2004
Posts: 162
Vogon poet
|
Vogon poet
Joined: Jan 2004
Posts: 162 |
Of course your first example is completely invalid because it has nothing to do with /filter. My claim was /filter -ww was as fast as (or only slightly faster than) /filter -ff/-wf/-fw, not that many $read's would be as fast as many memory accesses. That is a completely different function with completely different behaviour. My answer was equally related to your 'claim' as your 'claim' was related to mine. I say I want A and B and you pick out B and ignore A. If you would work in a MacDonalds, and they ask you a burger with cheese, are you gonna give the burger alone? The good thing about your example is that I can still do it with /filter using the -k switch to /write in a callback to some aux file and then merge it back after. I'll leave it as an excercise to you, but I guarantee it's not substantially slower than an $fline loop.
$fline() has nothing to do with the looping. It was 1000 times a $fline() search for the same line. The loop was there to get a benchmark reading with a higher resolution. You should better exercise yourself in reading what people want. The other good thing about your example though, is that even without /filter, there is still a faster way to do it.. it's called hash tables, which is mIRC's "storage method similar to custom windows" that you so incorrectly claim it lacks. Quickly converting the script to a hash table script,
alias yay { echo -ag Hash: $yayF @window: $yay@ }
alias yayF {
var %i = 1 | while (%i < 1001) { hadd -m yay %i a r g %i v 0 | inc %i }
var %t = $ticks, %i = 1
while (%i < 1001) {
var %rec = $hget(yay, $hfind(yay,& & & 500 & &,1,w).data)
hadd yay 500 $puttok(%rec,500,4,32)
inc %i
}
hfree yay
return $calc($ticks - %t)
}
alias yay@ {
var %i = 1 | window -h @yay | while (%i < 1001) { aline @yay a r g %i v 0 | inc %i }
var %t = $ticks, %i = 1
while (%i < 1001) {
var %rec = $fline(@yay,& & & 500 & &,1).text
rline @yay 500 $puttok(%rec,500,4,32)
inc %i
}
window -c @yay
return $calc($ticks - %t)
}
you find that: Hash: 94 @window: 328 So much for faster, again. So much for keeping its order. It was brought up earlier in this topic. Someone already mentioned this drawback. I confirmed it as relevant. Are you a politician or so? You seem to exhibit the same 'features'. A magnifying glass that makes you miss the bigger picture and blind for the obvious. Btw, the results here gave Hash: 220 @window: 480 Conclusion? Use the right tool for the right job. It really is that simple.[/code] Conclusion? You ignore selectively the requirements I gave. First you changed the case to usage of /filter alone. Here you limited the case to not keep order. What next?
[quote=argv0] edit: PS. I won't even bother deciphering your second example, but your initial statement is also wrong there. /filter can deal with strings like "*!*@*blah".. use the -g switch to make the matchtext a regular expression and then write it as: "\*!\*@\*blah". From what I understand, it makes your entire second alias completely obsolete.
About all mIRC's search commands support regex. Glad you assumed I didn't know that. Glad you didn't bother to think one step further. Glad you didn't bother to read the url to the other topic abit, instead of wildly chopping around with an axe. You might have had an idea why I didn't see regex as an option. But that's a notdone eh? Baffling around is so much easier - even more - it looks like it's your goal.
|
|
|
|
Joined: Sep 2005
Posts: 2,881
Hoopy frood
|
Hoopy frood
Joined: Sep 2005
Posts: 2,881 |
I created a bunch of array snippets that used hash tables to store sorted information. Whilst they are unsorted by nature, if you just use numbers as key names then you can manage the sort yourself. For example, if you were to sort this data alphabetically: z b d h j e, it would become: b d e h j z, you could then add each letter to the hash table with a number as key: hadd -m table 1 b
hadd -m table 2 d
hadd -m table 3 e
hadd -m table 4 h
hadd -m table 5 j
hadd -m table 6 z Then: echo -a $hget(table,1) = b
echo -a $hget(table,2) = d
echo -a $hget(table,3) = e
echo -a $hget(table,4) = h
echo -a $hget(table,5) = j
echo -a $hget(table,6) = z And there you have it, sorted hash tables The only thing you need to keep in mind when using this method is that if you were to delete an item you'd have to reshuffle every item after that position so there are no gaps, unless you come up with a decent method to iterate through the table until all items have been found even if there are gaps. This is where the main overhead comes in, it's quite slow to reshuffle 1,000+ items.
|
|
|
|
Joined: Oct 2003
Posts: 3,918
Hoopy frood
|
Hoopy frood
Joined: Oct 2003
Posts: 3,918 |
Wow someone's getting defensive. I say I want A and B and you pick out B and ignore A. To be fair, I didn't *ignore* anything. I ported your alias *command for command* to hash tables. It works exactly the same as your code, does exactly the same thing, and it's faster. So are you going to admit it's faster yet or are you going to cry about a bad experience at McDonalds? Btw, the results here gave Hash: 220 @window: 480 And your point is? That I'm still right? Thanks. Sorry when I tested the code I stepped off the "all computers run at exactly the same speed" planet. So much for keeping its order. I'm not sure who "confirmed" this (I don't see anyone confirming it here).. Hash tables items are not ordered in memory, yes, but if you practice what you preach and look at the code you'll see that my key names are in fact unique numerical indexes. Do you know what unique numerical indexes do when used as keys? They maintain order. $hget(yay,500) would get line 500, $hget(yay,200) would get line 200, etc.. I'm assuming you didn't actually understand this fact, or you wouldn't have made the above statement. To clarify, my hash table data *is* ordered, just not in your ram. Do you still think I'm wrong? About all mIRC's search commands support regex. Glad you assumed I didn't know that. Glad you didn't bother to think one step further. Huh? I didn't assume anything. I'm responding to your incorrect statement based on what you said here, namely: ...if you want to search for a line that contains *!bla@yay.yo in a certain token position, false positives are possible due to /filter's seeing it as a wildcard symbol This is simply wrong. First of all, they're not considered "false positives" if you incorrectly use a '*' as a literal when you know mIRC will parse it as a wildcard. That's called: using a command incorrectly. If you want to search for a line that contains '*' you use a regular expression match and escape it so it's not treated in a special manner. What exactly was I assuming here? I'm merely telling you that you CAN use /filter to match literal '*'s. Are you still saying you can't? Do you still think I'm wrong? You don't seem to enjoy factual proof. I'm giving you working code with improved benchmark results that you yourself corroborate and for some reason my giving of this information makes you call me a "politician", which makes no sense to me. Rather than focusing on why my facts may be invalid you're more interested in horrible analogies and trying to insult me by calling me names. Real classy. I'm not sure how else I can convince you...
- argv[0] on EFnet #mIRC - "Life is a pointer to an integer without a cast"
|
|
|
|
Joined: Jan 2004
Posts: 162
Vogon poet
|
Vogon poet
Joined: Jan 2004
Posts: 162 |
I created a bunch of array snippets that used hash tables to store sorted information. Whilst they are unsorted by nature, if you just use numbers as key names then you can manage the sort yourself. For example, if you were to sort this data alphabetically: z b d h j e, it would become: b d e h j z, you could then add each letter to the hash table with a number as key: hadd -m table 1 b
hadd -m table 2 d
hadd -m table 3 e
hadd -m table 4 h
hadd -m table 5 j
hadd -m table 6 z Then: echo -a $hget(table,1) = b
echo -a $hget(table,2) = d
echo -a $hget(table,3) = e
echo -a $hget(table,4) = h
echo -a $hget(table,5) = j
echo -a $hget(table,6) = z And there you have it, sorted hash tables The only thing you need to keep in mind when using this method is that if you were to delete an item you'd have to reshuffle every item after that position so there are no gaps, unless you come up with a decent method to iterate through the table until all items have been found even if there are gaps. This is where the main overhead comes in, it's quite slow to reshuffle 1,000+ items. In the past I've seen similar ideas. I know most problems have a solution. But I'll recap what I said: My interest in this * Error creating window (and * Error creating dialog) is because in ran in both, the first when having alot connections/channels and the latter when I try to test my script between two mIRC instances (it has a kind of remote control through sockets, where remote configuration files are sent and loaded in quite big dialogs). Beside this, my script uses an amount custom windows for script configuration data, but that amount doesn't increase with the amount irc connections so it's not a problem as worse as the irc windows. I found the way custom windows can be used very handy. They keep their order by themselve, they are fast and various powerful commands work on them, /filter can be used to sort on token position, $fline() is handy to find a line with a certain token on a certain position, and several other commands provide in add/delete/insert/etc of one single line. So, for every sort of task, there's a choice. The big 'but' of this is that the window-being of this storage solution is actually an unwanted side-effect. mIRC should have a similar memory structure, maybe in the form of an array where the 'slot'-names are 'behind the scenes', leaving only the value part. I don't know if such structure already exists under some other name. And of course, also commands similar to what now is available for custom windows. Clickhere once said that a dll could mimic this easily and wouldn't be hard to write, but I'm not a big fan of using dll's for such script core functionality, they create a heavy dependency on a completely third party-'plugin' to a main commercial application. If for any reason the dll becomes unusable then the mIRC script completely relying on it follows it into the grave. Back to the question I had, suppose this * Error creating window stops a channel window from being created, could it be so programmed that mIRC can be told to not create a window (thus on purpose)? Because that would allow scripts to be on alot networks/channels, unlimited by this resource problem, while still be able to use all what mIRC has available as functions, because the only alternative is to use socket connections to the irc servers, inflicting the need to do -everything- in script, from protocol parsing till IAL, notify, etc. The question is if this is realistic, as is, how much are mIRC's functions dependent on the existence of the window.
|
|
|
|
Joined: Jan 2004
Posts: 162
Vogon poet
|
Vogon poet
Joined: Jan 2004
Posts: 162 |
Wow someone's getting defensive. I started to talk to you like you talk to me. Like it? To be fair, I didn't *ignore* anything. I ported your alias *command for command* to hash tables. It works exactly the same as your code, does exactly the same thing, and it's faster. So are you going to admit it's faster yet or are you going to cry about a bad experience at McDonalds? And your point is? That I'm still right? Thanks. Sorry when I tested the code I stepped off the "all computers run at exactly the same speed" planet. I'm not sure who "confirmed" this (I don't see anyone confirming it here).. Hash tables items are not ordered in memory, yes, but if you practice what you preach and look at the code you'll see that my key names are in fact unique numerical indexes. Do you know what unique numerical indexes do when used as keys? They maintain order. $hget(yay,500) would get line 500, $hget(yay,200) would get line 200, etc.. I'm assuming you didn't actually understand this fact, or you wouldn't have made the above statement. To clarify, my hash table data *is* ordered, just not in your ram. Do you still think I'm wrong? Huh? I didn't assume anything. I'm responding to your incorrect statement based on what you said here, namely: This is simply wrong. First of all, they're not considered "false positives" if you incorrectly use a '*' as a literal when you know mIRC will parse it as a wildcard. That's called: using a command incorrectly. If you want to search for a line that contains '*' you use a regular expression match and escape it so it's not treated in a special manner. What exactly was I assuming here? I'm merely telling you that you CAN use /filter to match literal '*'s. Are you still saying you can't? Do you still think I'm wrong? You don't seem to enjoy factual proof. I'm giving you working code with improved benchmark results that you yourself corroborate and for some reason my giving of this information makes you call me a "politician", which makes no sense to me. Rather than focusing on why my facts may be invalid you're more interested in horrible analogies and trying to insult me by calling me names. Real classy. I'm not sure how else I can convince you...
I throwed all your text together on purpose. I already told you: I'm not using regular expressions. For the rest, arguev0 further with yourself about usage of terms and statements and blah blah blah.
|
|
|
|
Joined: Sep 2005
Posts: 2,881
Hoopy frood
|
Hoopy frood
Joined: Sep 2005
Posts: 2,881 |
I'd love to see an array structure with similar implementation to hashtables.
/amake <name> /aadd [-m] <name> [index] <data> ** if [index] is specified it will overwrite that index. /adel <name> <index> /ainsert [-m] <name> <index> <data> ** [-m] means the same as it does with hashtables. Create array if it doesn't already exist. /afree <name> $aget(name,index) $afind(name,text[,nwWrR])[.data]
With an array being a list that maintains order then obviously insert/overwrite commands would be beneficial too. Then a switch for /filter to act on arrays would be good.
|
|
|
|
Joined: Oct 2003
Posts: 3,918
Hoopy frood
|
Hoopy frood
Joined: Oct 2003
Posts: 3,918 |
"I'm not using regular expressions"?
What is that, some kind of protest?
No mommy, I refuse to eat my lunch!
The fact that you're making your own life miserable is completely beside the point that it's possible.
- argv[0] on EFnet #mIRC - "Life is a pointer to an integer without a cast"
|
|
|
|
Joined: Jul 2006
Posts: 4,180
Hoopy frood
|
Hoopy frood
Joined: Jul 2006
Posts: 4,180 |
I'd love to see an array structure with similar implementation to hashtables.
/amake <name> /aadd [-m] <name> [index] <data> ** if [index] is specified it will overwrite that index. /adel <name> <index> /ainsert [-m] <name> <index> <data> ** [-m] means the same as it does with hashtables. Create array if it doesn't already exist. /afree <name> $aget(name,index) $afind(name,text[,nwWrR])[.data]
With an array being a list that maintains order then obviously insert/overwrite commands would be beneficial too. Then a switch for /filter to act on arrays would be good. I've just seen your ideas, what's the difference with what you suggest and hash table method ? I've don't see any difference, it does exactly the same thing, just have to suggest /hins [-m] <name> <item> <data>, assuming that you're using number as item (would be a nice idea). However, the switch for /filter could be very useful, how you think it would work ? I have an idea, but just to see if it's the same, this could be very very useful as well
#mircscripting @ irc.swiftirc.net == the best mIRC help channel
|
|
|
|
Joined: Dec 2002
Posts: 2,962
Hoopy frood
|
Hoopy frood
Joined: Dec 2002
Posts: 2,962 |
An array is an ordered structure where each item is indexed by an integer (ie. its position within the array), a hash table is an unordered structure where each item is indexed by a key. The majority of the commands and identifiers associated with each might look the same but the situations for which they are suitable differ considerably.
Spelling mistakes, grammatical errors, and stupid comments are intentional.
|
|
|
|
Joined: Jul 2006
Posts: 4,180
Hoopy frood
|
Hoopy frood
Joined: Jul 2006
Posts: 4,180 |
a hash table is an unordered structure where each item is indexed by a key Sure, but if each item is a number, then it's ordered, what would be the difference between using an hash table with number index for each item name and hixxy's feature ?
#mircscripting @ irc.swiftirc.net == the best mIRC help channel
|
|
|
|
Joined: Apr 2004
Posts: 759
Hoopy frood
|
Hoopy frood
Joined: Apr 2004
Posts: 759 |
With hash tables adding them in order does not mean their stored sorted numerically based on their key value as well. Their stored sorted on their hashes. The following code demonstrates this:
//var %x = 1 | while (%x < 100) { hadd -m th %x hello %x | inc %x } | hdel th 50 | echo -a $hget(th,50) => $hget(th,2).item | hfree th
$hget(th,50) does not echt hello 51 it just doesn't excist, In arrays slicing a piece out (hdel) would bump the rest back up. $hget(th,2).item does not return 2 but 4. This is the major advantage of non-ascociative arrays over hashtables. I'd love to see native array support although i rather see a pop/push/slice etc terminology implemented over hixxy's suggestion.
$maybe
|
|
|
|
Joined: Jul 2006
Posts: 4,180
Hoopy frood
|
Hoopy frood
Joined: Jul 2006
Posts: 4,180 |
I know how hash table work, and I was needed some little clarification about what hixxy suggested, you've done this job thanks, and then indeed, it would be very useful, should be added as a feature suggestion if it's not already there.
#mircscripting @ irc.swiftirc.net == the best mIRC help channel
|
|
|
|
Joined: Oct 2003
Posts: 3,918
Hoopy frood
|
Hoopy frood
Joined: Oct 2003
Posts: 3,918 |
As people pointed out, using indexed values for keys does not create an internally ordered structure, though the usage of the hash table API where keys are ordered integers *does* result in an ordered structure for all intensive purposes. On one hand, I see why certain nitpickers might care about the internal implementations that mIRC might use for efficiency. On the other hand, the *majority* of users should not be concerned with implementation, as that's Khaled's job... the fact that you can do: hadd tab 2 item | $hget(tab,2) => item means it's functionality equivalent to an array API, so I don't see the issue from a superficial standpoint.
Was this suggested specifically for efficiency reasons, or for functionality? If it really is an efficiency thing, would this really make a big difference? I doubt the bottleneck is the hash function, and I'd bet there would be no noticeable performance improvement, though benchmarks would be interesting to see.
The most constructive thing I could suggest is to add a switch (-i?) on /hmake to index instead of hash. This would mean keys would need to be numeric but it would allow mIRC to implement the structure internally as an array while keeping the user level API the same. The benefit here would be not having to introduce an equivalent set of commands/identifiers that have the same functional behaiour.
- argv[0] on EFnet #mIRC - "Life is a pointer to an integer without a cast"
|
|
|
|
Joined: Dec 2002
Posts: 2,962
Hoopy frood
|
Hoopy frood
Joined: Dec 2002
Posts: 2,962 |
Hash tables aren't functionally equivalent even if you use numeric keys. Deletion may be equivalent as a byproduct of mIRC's implementation (I haven't checked) but insertion certainly won't be possible without manually shifting all higher indexes - now that definitely would result in unacceptable performance.
The hash table API also isn't very descriptive when applied to arrays - for example would /hadd insert a new item at a given index or overwrite it? Adding support for slicing in relevant commands would add an unnecessary layer of complexity to the /h* commands making them quite different depending on what kind of 'hash table' was being used, not to mention there would also (presumably) be additional commands for arrays that would have no relevance to hash tables (eg. sorting). All in all having two different concepts sharing a set of commands would be extremely messy and would only confound most scripters.
Spelling mistakes, grammatical errors, and stupid comments are intentional.
|
|
|
|
Joined: Apr 2004
Posts: 759
Hoopy frood
|
Hoopy frood
Joined: Apr 2004
Posts: 759 |
Associative arrays map very wel to hash tables. Pure arrays don't, i'm with starbucks_mafia on this one. (Deletion is not a byproduct either see code i posted above, it will create unreferencable gaps.) the fact that you can do: hadd tab 2 item | $hget(tab,2) => item means it's functionality equivalent to an array API, so I don't see the issue from a superficial standpoint.
This holds truth provided you dont insert and delete items of the 'stack'. Pure arrays will be used as stacks where inserting and/or deleting items will re-order the stack accordingly automatically so they're always in a loop safe state. In short hash table is great for associative data but a mess to loop through (Since gaps CAN appear). Pure arrays have nothing to do with associative data but are excellent to loop trough. Since they are not the same thing i dont think if arrays are ever implemented that they should share the same terminology as the hash table commands let alone share the commands
Last edited by Mpdreamz; 11/02/09 05:36 PM.
$maybe
|
|
|
|
Joined: Sep 2005
Posts: 2,881
Hoopy frood
|
Hoopy frood
Joined: Sep 2005
Posts: 2,881 |
alias fakearray {
hadd -m array 1 a
hadd -m array 2 b
hadd -m array 3 c
hdel array 2
var %i = 1
while ($hget(array,%i) != $null) {
echo -a $v1
inc %i
}
} Here is one of the major problems with using a hash table to simulate an array. The only way to fix the problem with the gaps, is to shift all items past the point of deletion down. This takes some amount of time on big tables: I know because I wrote a script that for all intents and purposes lets you create perfect arrays using hash tables. I don't have the full script anymore, but I've whipped up a little test script to give you some idea of how slow it is to shift down entries in a hash table when you need to delete a numerical index. alias -l create {
var %i = $2
while (%i) {
hadd -m $1 %i %i
dec %i
}
}
alias -l delete {
hdel array $2
var %i = $2 + 1
while ($hget($1,%i) != $null) {
hadd -m $1 $calc(%i - 1) $v1
inc %i
}
hdel $1 $hget($1,0).item
}
alias testspeed {
var %ticks = $ticks
create array 3000
var %i = 1500, %j = 1400
while (%i > %j) {
delete array %i
dec %i
}
hfree array
echo -a $calc($ticks - %ticks) ms.
} This creates a hash table containing 3,000 items, then it deletes 100 of those items from somewhere in the middle of the table. Each time it does this, it shifts all of the indexes above that down, so that the "array" maintains order (and can still be referred to as $hget(array,N)). On my 3ghz dual core CPU this takes around 4.6 seconds to execute, and it's not even an unrealistic example. Having arrays built in would be much more efficient and also convenient.
|
|
|
|
|