mIRC Home    About    Download    Register    News    Help

Print Thread
Deleting matching wildcards in hash tables #219698 25/03/10 11:05 PM
Joined: Mar 2009
Posts: 74
K
KageNoOni Offline OP
Babel fish
OP Offline
Babel fish
K
Joined: Mar 2009
Posts: 74
I am unfortunately not familiar with the use of hash tables at all, and need an efficient method to go through every item on the table (it's rather large) and delete specific data items, and if the one deleted was the last item for that entry, delete the entry itself (if that isn't handled automatically by mIRC). Anyone able to help with this?

I got a script at one point, that keeps track of all nicks a person with a specific ident and isp uses, but in on ustream, a lot of people end up joining a ustreamer-#####, and rejoining with a new one using a randomized number. At this point many of the lines are so long I'm getting an error message.
* /hadd: line too long

Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #219700 25/03/10 11:31 PM
Joined: Nov 2006
Posts: 1,559
H
Horstl Offline
Hoopy frood
Offline
Hoopy frood
H
Joined: Nov 2006
Posts: 1,559
For the matching-&-deleting, provide some sample data please (say, three item=data lines).
So far, it sounds like a task for a loop of $hfind(table,wildstring,N,w).data to wildmatch the data (whole data line) of all items in question and a further loop of $deltok(line,$wildtok(line,wildstring,N)) on each of these data lines (to find and remove matching tokens/words from the data)...
mIRC won't delete "empty" items from a hash table automatically, but adding a check - some "if ($hget(table,item) == $null)" - won't add much overhead to that a script.

As a general design idea and to avaid that error in general, why not check the total data length every time you add a fresh chunk of data to an item; and drop some older data after a certain limit had been reached (e.g. cut off some of the first words/tokens before adding new data at the end)?

Re: Deleting matching wildcards in hash tables [Re: Horstl] #219701 26/03/10 12:01 AM
Joined: Mar 2009
Posts: 74
K
KageNoOni Offline OP
Babel fish
OP Offline
Babel fish
K
Joined: Mar 2009
Posts: 74
Much of the problem comes from not knowing how to use hash tables at all, this is really just a script some one else wrote. I don't know how much can be added before it starts giving that error message, so I really don't know what maximum I should check for.

And the 3 lines you suggested including, did you want lines of the .dat file? Or lines from some part of the script? I think you mean the .dat file, so I'll show you that.

Code:
ustreamer-30244 CrazyArtist
*!*streamer-@*.roose1.nb.home.nl
ustreamer-31240 ustreamer-87712
*!*mer-55049@*.hsd1.pa.comcast.net
ustreamer-55049
*!*streamer-@*.ca.shawcable.net
spangus ustreamer-25449 ustreamer-5733 ustreamer-32732 ustreamer-94722 ustreamer-36823 ustreamer-67198


Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #219703 26/03/10 01:14 AM
Joined: Oct 2004
Posts: 8,330
Riamus2 Offline
Hoopy frood
Offline
Hoopy frood
Joined: Oct 2004
Posts: 8,330
The length of the item depends on the version of mIRC you use. It was increased a lot in 6.35. Not sure what it is, but it's probably in versions.txt ... maybe 4096 characters or something?

One thing to consider... are hash tables your best option for what you're doing? Is the data used all the time or just occasionally? If it's not used that often, a text file or some other method might be better.


Invision Support
#Invision on irc.irchighway.net
Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #219704 26/03/10 01:39 AM
Joined: Nov 2006
Posts: 1,559
H
Horstl Offline
Hoopy frood
Offline
Hoopy frood
H
Joined: Nov 2006
Posts: 1,559
At first I was thinking about $hfind(w), but with the w-method you'd likely wildmatch "lines" (whole data-blocks) even if they don't contain a specific "word" that's wildmatching your search term. I therefore used $hfind(r) aka a regex search. It should perform more efficient in this case (at least that's what I was after :D).
As a big downside, the code is of very little use if you want to learn about hash tables in general. Feel free to ask questions tho... I generally suggest to look for other scripts that use hash tables - you can learn a lot if you study real code after the helpfile smile

Code:
; syntax: /delhashtokens *a*wildc?rd-expres*n

alias delhashtokens {
  ; put the name of your hash table here 
  var %t = TABLENAME

  ; create a regular expression ouf of the "wildcard"-word $1.
  ; the expression should only match lines that contain any "word" wildmatching your expression - in contrast to a "line" wildmatching your expression.
  ; ( it quotes $1 so no text char is treated as "metachar", replaces the wildcards */? with regex-equivalents, and adds lookarrounds to match like a $istok(,32) )
  var %ex = $+(/,(?<=^|\s)\Q,$replacexcs($$1,*,\E\S*\Q,?,\E\S\Q,\E,\E\\E\Q),\E(?=\s|$),/)
  ; loop all items matching the regular expression 
  var %n = $hfind(%t,%ex,0,r).data
  while (%n > 0) {
    ; the current item and it's data
    var %item = $hfind(%t,%ex,%n,r).data, %data = $hget(%t,%item)
    ; find and remove all the "words" from the data that wildmatch the original expression
    while ($wildtok(%data,$1,1,32)) { var %data = $deltok(%data,$findtok(%data,$v1,1,32),32) }
    ; delete item if no data is left or save the new data
    $iif((%data == $null),hdel,hadd) %t %item %data
    ; proceed with next item
    dec %n
  }
}
Side note: backup your .dat file first, I didn't test this at length smile

About limiting the data length to avoiod the errror in the future, I can give only an example so you might get the idea...
Code:
; original code example

; adds a new word to the data by overwriting it with "current data + new word" (the new word being "$nick")
hadd TABLE ITEM $hget(TABLE,ITEM) $nick


; new code example

; if current data exceeds a certain length: remove the "oldest" (first) 4 words by re-adding only the 5th word onwards together with the new word
var %data = $hget(TABLE,ITEM)
if ($len(%data) > 4000) { var %data = $gettok(%data,5-,32) }
hadd TABLE ITEM %data $nick


Edit: shorter expression

Re: Deleting matching wildcards in hash tables [Re: Riamus2] #219705 26/03/10 01:39 AM
Joined: Mar 2009
Posts: 74
K
KageNoOni Offline OP
Babel fish
OP Offline
Babel fish
K
Joined: Mar 2009
Posts: 74
Any time some one joins a chat I'm in, or changes a nick, as well as when I first join, the info in the hash tables are accessed and possibly modified. This can happen hundreds of times in joining one single chat (the /who command). The chat where I'm getting the message about the size of the data table in particular has people joining, leaving, and changing nicks, almost constantly. One channel I'm on has only 40 people and has nick changes even more frequently (it's the chat for an mmo, any time some one changes characters, their irc nick changes to match that character). The data needs to be readily available for modification constantly, so yes, this the hash tables are probably the best method for this.

I keep my version of mIRC updated, so I do have the extra line length. Unfortunately though, it doesn't seem to be enough, due to this chat creating so many alternate nick names for people that aren't really alternates, and as a result clogging up the .dat file where the info is saved (when I close mIRC only).

Edit: Horstl, thanks for that, I'll make that backup and test this out in a second, but more useful to me than that limit, would be just a way to prevent those ustreamer nicknames from being added to the tables in the first place.

Last edited by KageNoOni; 26/03/10 01:42 AM.
Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #219707 26/03/10 01:59 AM
Joined: Nov 2006
Posts: 1,559
H
Horstl Offline
Hoopy frood
Offline
Hoopy frood
H
Joined: Nov 2006
Posts: 1,559
Well, add conditions to all the events where nicks are added (on join, on nick, end of who, ...)
Code:
; at the nick event
    if (ustreamer-* !iswm $newnick) { -nick adding- }
; at the join event
    if (ustreamer-* !iswm $nick) { -nick adding- }
; etc

Maybe you're lucky and your script uses a distinct routine (alias) for this - so you have to put the extra condition only at one place.

Re: Deleting matching wildcards in hash tables [Re: Horstl] #219708 26/03/10 02:09 AM
Joined: Mar 2009
Posts: 74
K
KageNoOni Offline OP
Babel fish
OP Offline
Babel fish
K
Joined: Mar 2009
Posts: 74
It does not use a distinct routine sadly. Every script I use that might require a routine used in more than one place, gets an alias with the routine. In fact, I often make such routines anyway just to give me the freedom to do that in the future if I decide I may want to use it else where later. Those routines greatly increase readability of a script anyway.

Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #219710 26/03/10 02:14 AM
Joined: Mar 2009
Posts: 74
K
KageNoOni Offline OP
Babel fish
OP Offline
Babel fish
K
Joined: Mar 2009
Posts: 74
It worked! Thanks, though it took quite a while. It turns out there were over 4,000 matches, I added a quick /echo to see how many matches there were. A quick /hsave found all the ustreamer nicks had been removed.

Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #219721 26/03/10 02:31 PM
Joined: Nov 2006
Posts: 1,559
H
Horstl Offline
Hoopy frood
Offline
Hoopy frood
H
Joined: Nov 2006
Posts: 1,559
Daft of me to use a decreasing loop. It will consume most of the computing time to loop to $hfind(,N) - especially for a high N value. Too late now... but pro forma, it should better be:
Code:
alias delhashtokens {
  var %t = TABLENAME, %ex = $+(/,(?<=^|\s)\Q,$replacexcs($$1,*,\E\S*\Q,?,\E\S\Q,\E,\E\\E\Q),\E(?=\s|$),/)
  while ($hfind(%t,%ex,1,r).data != $null) {
    var %i = $v1, %d = $hget(%t,$v1)
    while ($wildtokcs(%d,$1,1,32)) { var %d = $deltok(%d,$findtokcs(%d,$v1,1,32),32) }
    $iif((%d == $null),hdel,hadd) %t %i %d
  }
}

Re: Deleting matching wildcards in hash tables [Re: Horstl] #219729 26/03/10 10:12 PM
Joined: Mar 2009
Posts: 74
K
KageNoOni Offline OP
Babel fish
OP Offline
Babel fish
K
Joined: Mar 2009
Posts: 74
Thanks for that, I've updated the command, but hopefully my check to stop processing the script for ustreamer nicks will make this no longer necessary.

By the way, why did you use 'if (<condition>) { script }' instead of 'if (!<condition>) { halt }'? If something evaluating to 'true' or 'false' would cause a script to not need to be run, I find it's easier to keep track of how nested I am into a script if I don't have the entire script inside yet another if statement.

Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #219730 26/03/10 10:35 PM
Joined: Aug 2004
Posts: 7,252
R
RusselB Offline
Hoopy frood
Offline
Hoopy frood
R
Joined: Aug 2004
Posts: 7,252
One reason for using 'if (<condition>) { script }' instead of 'if (!<condition>) { halt }' is when there's a possibility of a valid response having a 0 (zero) value.

The !<condition> registers as true when <condition> is $false, $null or 0

I didn't look through the script to see if this might be the reason or not, but it's a possibility.

Additionally, if you had a lot of 'if (!<condition>) { halt }' statements, it's possible that not only that script, but other scripts could be unintentionally halted.

Re: Deleting matching wildcards in hash tables [Re: RusselB] #219818 30/03/10 03:07 AM
Joined: Mar 2009
Posts: 74
K
KageNoOni Offline OP
Babel fish
OP Offline
Babel fish
K
Joined: Mar 2009
Posts: 74
Yeah, I suppose that's possible, but neither of those would have been possible in this situation. The check was to see if certain nicks were being used, and skip the script if there was a match of the wildcard, and the nick. If it is a match, skip the script, otherwise continue, so it seems easier to just use a /halt in this situation, and makes it easier to keep track of how nested something is in the script.

Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #219825 30/03/10 12:46 PM
Joined: Nov 2006
Posts: 1,559
H
Horstl Offline
Hoopy frood
Offline
Hoopy frood
H
Joined: Nov 2006
Posts: 1,559
It's up to you whether you use in this particular case (nest the code under some condition, or have some condition stop processing of the following code). smile But if you decide to use the latter, mind the different effects of /halt vs. /return on events and calling aliases (this thread could be of help; /halt can also affect the &-event prefix and $halted)

Re: Deleting matching wildcards in hash tables [Re: Horstl] #219861 01/04/10 03:49 AM
Joined: Mar 2009
Posts: 74
K
KageNoOni Offline OP
Babel fish
OP Offline
Babel fish
K
Joined: Mar 2009
Posts: 74
I've never needed to worry about $halted or the & prefix, how ever I am very familiar with the differences between 'return' and 'halt', as I use both in a variety of scripts. Neither are really the case here, and I won't be calling the custom /delhashtokens alias from another script, so using halt here won't be a problem.

Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #219876 01/04/10 09:13 PM
Joined: Feb 2009
Posts: 133
C
chacha Offline
Vogon poet
Offline
Vogon poet
C
Joined: Feb 2009
Posts: 133
i just read the topic but i dont understand :p
please can you explain me what's the point of this code?


WorldDMT
Re: Deleting matching wildcards in hash tables [Re: chacha] #219998 03/04/10 11:11 PM
Joined: Mar 2009
Posts: 74
K
KageNoOni Offline OP
Babel fish
OP Offline
Babel fish
K
Joined: Mar 2009
Posts: 74
Originally Posted By: chacha
i just read the topic but i dont understand :p
please can you explain me what's the point of this code?


The original script keeps track of every time a person changes a nickname, and stores them in a hash table. It keeps uses $address($nick,3) to keep track of which set of nicknames have been used by any given person. The point of the thread was finding an efficient way of going through the listed nicknames, and delete the ones that I have no use for, and prevent them from being added any more, so as not to fill my hash table with useless data, preventing it from remebering the nicks I want it to remember.

Re: Deleting matching wildcards in hash tables [Re: KageNoOni] #220070 04/04/10 11:07 PM
Joined: Feb 2009
Posts: 133
C
chacha Offline
Vogon poet
Offline
Vogon poet
C
Joined: Feb 2009
Posts: 133
can u plz show us your code


WorldDMT