mIRC Home    About    Download    Register    News    Help

Print Thread
Page 1 of 3 1 2 3
#199058 08/05/08 03:35 PM
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
What's the difference between hsh and txt files? Is one better then the other? If so, why is it better?


I registered; you should too.
Joined: Oct 2004
Posts: 8,330
Hoopy frood
Offline
Hoopy frood
Joined: Oct 2004
Posts: 8,330
Nothing but the extension. You can save data using any extension you like... even ones used by other programs (such as .avi). It doesn't really matter, though it could be confusing to use something that another program uses. It's the contents that make the difference... not the extension.

Often, people use .hsh when saving hash tables just so that they know it's hash table data, but that's entirely up to the individual.


Invision Support
#Invision on irc.irchighway.net
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Why do people use hash files much more then text files? Is there a benefit? mIRC /help hash says "Hash tables allow you to efficiently store large amounts of information which can be quickly referenced and retrieved later on." You can store large amounts of information with text files too. So what are the pro's and con's of each kind?


I registered; you should too.
Joined: Aug 2004
Posts: 7,252
R
Hoopy frood
Offline
Hoopy frood
R
Joined: Aug 2004
Posts: 7,252
Hash tables are stored in RAM
Files are stored on your hard drive, so they are speed limited by the speed of your hard drive. Even the fastest drives that I know of, 10000 rpms, are so much slower than the slowest ram, that it's almost not worth bothering with a test.

If you want to see a test, I can do one later on a customized system built to those specifications.

Joined: Dec 2002
Posts: 2,962
S
Hoopy frood
Offline
Hoopy frood
S
Joined: Dec 2002
Posts: 2,962
The benefit isn't in the files. Once you've loaded a file into a hash table using the /hload command the data is stored in memory, which means it's several orders of magnitude faster than accessing a file directly each time.

Alternatively you could read the contents of a text file into multiple global variables in order to speed up access, but it's generally uglier in terms of code and you lose efficiency of the powerful built-in searching capabilities available to hash tables via $hfind.


Spelling mistakes, grammatical errors, and stupid comments are intentional.
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Okay, so I read up on the mIRC /help HASH

First, you /hmake

I dont understand "eg. if you expect that you'll be storing 1000 items in the table, a table of N set to 100 is quite sufficient."

Lets say I'm expecting 10,000 userhost. Why wouldn't the /hmake be set to 10,000?

Also, how can i visualize a hash table?

name info1 info2 info3
apple qwe wrtg sdgf
orange sdfg gdrty utgh

???


I registered; you should too.
Joined: Oct 2003
Posts: 3,918
A
Hoopy frood
Offline
Hoopy frood
A
Joined: Oct 2003
Posts: 3,918
If you set 10,000 you will be using 10 thousand times more memory than if you used.. say, 1000, which would be sufficient according to the docs.

Of course generally speaking we're only talking about 200kb to 2mb more memory (depending on the internal size of the hash table), but it's still plenty more. If you're making many of these tables they will add up.

The reason is a hash table expands dynamically.. there's some hash table theory behind the size of N that can be explained on google, it's a bit too heavy to go into detail here. The rule of thumb is "expected size divided by 10" if the expected size is big. If you know the exact size and it's relatively small, you can set that exactly.. like N=100

To visualize a table you can loop through it with $hget(name, %i) .. There is probably a snippet out there somewhere to do this... check http://www.mirc.net or http://www.mircscripts.org for that.


- argv[0] on EFnet #mIRC
- "Life is a pointer to an integer without a cast"
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
/hmake (creates table in memory)
/hload (loads file into memory - does this load the whole file?)
/hadd (to add/replace item
/hfind (to find an item)
/hget (to look up an item)
/hdel (to delete an item)
/hfree (to free an existing table from memory)
/hsave (to save a table from memory to file)

Do I understand this correctly?


I registered; you should too.
Joined: Jan 2006
Posts: 111
N
Vogon poet
Offline
Vogon poet
N
Joined: Jan 2006
Posts: 111
This is an old discussion and I can remember that I said that ini files are stored in memory too. See /help /flushini. I think working with ini files is much more easier than working with hsh files.

Joined: Oct 2003
Posts: 3,918
A
Hoopy frood
Offline
Hoopy frood
A
Joined: Oct 2003
Posts: 3,918
You'd be wrong if you said that.

ini files are *cached* by Windows but *not* necessarily in active memory. They're likely to be paged out onto disk, in fact. The file access cache is still slower than active memory. Working with ini files is not easier, by any means. The only caveat of hash files is using them in write-mode.. of course, /writeini doesn't cache at all, so hash tables are still more efficient, just a bit more work to save them periodically (if you need to write out changes).


- argv[0] on EFnet #mIRC
- "Life is a pointer to an integer without a cast"
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Okay I want to start out with making a very small hash file. I have a checkbot alias:


Code:
botcheck %nick %host %chan

alias botcheck {
  if ($istok(1.2.3.4 5.6.7.8 2.3.4.5 6.7.8.9 3.4.5.6 7.8.9.0, $2, 32)) {
    echo 4 -s $1 at $2 on $3
    echo 4 -s -
    msg $3 bot: $1
  }
  return
}


So to change it to use a hash table:

Code:
botcheck %nick %host %chan

alias botcheck {
  if ($hfind(bots, $2, 1)) {
    echo 4 -s $1 at $2 on $3
    echo 4 -s -
    msg $3 bot: $1
  }
  return
}


I registered; you should too.
Joined: Oct 2003
Posts: 3,918
A
Hoopy frood
Offline
Hoopy frood
A
Joined: Oct 2003
Posts: 3,918
AWEstun:

It's all in the help file. If you have a specific issue, feel free to ask the question.. but if you're asking us to read the help file to you then I don't see the point.

edit:

Your code has no hash table stuff in it. Why not start out by adding your hash table code and *then* asking if it's right/wrong? ...Actually, you only really need to ask if it's wrong smile

Last edited by argv0; 08/05/08 06:51 PM.

- argv[0] on EFnet #mIRC
- "Life is a pointer to an integer without a cast"
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Please don't be SO FORWARD. I am reading the mIRC /help and I have questions. I appreciate all the help from people. I want to learn how to start using hash tables.

This is in the remote.ini file:

Code:
botcheck %nick %host %chan

alias botcheck {
  if ($hfind(bots, $2, 1)) {
    echo 4 -s $1 at $2 on $3
    echo 4 -s -
    msg $3 bot: $1
  }
  return
}


This is in the alias.ini file:

Code:
/mbot //hmake -s bots 1
/mbots //hmake -s bots 1
/lbot //hload -s bots bots.hsh
/addbot //hadd -s bots $$1
/delbot //hdel -s bots $$1
/sbots //hsave -s bots bots.hsh
/freebot //hfree -s bots


I registered; you should too.
Joined: Oct 2003
Posts: 3,918
A
Hoopy frood
Offline
Hoopy frood
A
Joined: Oct 2003
Posts: 3,918
You never want to use "1" as the size of the hash table, you should really use a minimum of 100. I forgot to mention that.. the "rule of thumb" applies to large values, not small ones.

You also should design your htable to make as little use of $hfind as possible..

$hget(name, ITEM) is the fastest way to access data associated with ITEM (it's the point of hash tables after all).. Try to use $hget to access your data unless you really need to *search*.. I doubt you do.

Do you have any specific issues with the code? I'm not sure what else to say... Like I said, you get better answers if you ask specific questions.


- argv[0] on EFnet #mIRC
- "Life is a pointer to an integer without a cast"
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
So instead of $hfind, I can use $hget, like this?

Code:
alias botcheck {
  if ($hget(bots, $2)) {
    echo 4 -s $1 at $2 on $3
    echo 4 -s -
    msg $3 bot: $1
  }
  return
}


I registered; you should too.
Joined: Oct 2003
Posts: 3,918
A
Hoopy frood
Offline
Hoopy frood
A
Joined: Oct 2003
Posts: 3,918
It depends. is $2 the key?

Question: Do you know how ini files work? Because hash tables work exactly the same way.

An ini file is like:

[section]
key=value

a hash table is like:

[tablename]
key=value

But in memory.. not a file

So $hget(tablename, key) would be value, just like $readini(file, section, key)



- argv[0] on EFnet #mIRC
- "Life is a pointer to an integer without a cast"
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
$2 is the host which is the IP: 1.2.3.4

Haven't used hash or ini tables until now.

I'm just starting to learn how hash tables work.


I registered; you should too.
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
How would I 'echo' out the contents of of a table with a echo line command at the command promt?

Somethings not right with that $hget command. When I tried:

if ($hget(bots,$2)) {

where $2 is the current user's host, it wasn't executing for the hosts inside the bots table. I saw several of the bot hosts in the bots table not tribber.

When I put:

$hfind(bots,$2,1)

it started working again.


I registered; you should too.
Joined: Aug 2004
Posts: 7,252
R
Hoopy frood
Offline
Hoopy frood
R
Joined: Aug 2004
Posts: 7,252
You probably wouldn't want to echo the entire contents of the table, but if you did, here's what you would use from the command prompt. Replace all occurances of <table> with the name of the hash table created using the /hmake command.

Code:
//var %a = 1, %b = $hget(<table>,0).item | while %a <= %b { echo -a $hget(<table>,%a).item is $hget(<table>,%a) | inc %a }

Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Thanks. I just wanted to verify the 6 hosts in the table. They are there.

Do you have any idea why the $hfind works for me, but the $hget doesn't?


I registered; you should too.
Joined: Jul 2006
Posts: 3,971
W
Hoopy frood
Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,971
It's probably because you don't use a value when adding a item ($$1) to the hash table :
Quote:
/addbot //hadd -s bots $$1
so $hget(table,$2) return the value, which is $null.Your $hfind work because it search in table for an item, not a value (you can search for value with the propriety .data, read the help file)

Edit : In the loop of RusselB, only $hget(<table>,%a).item will return something.You should use something like :
Code:
/addbot //hadd -s bots $$1 1
With this, $hget(bots,host) will return 1 if the host exist, and $null if doesn't.


Last edited by Wims; 09/05/08 12:19 AM.

#mircscripting @ irc.swiftirc.net == the best mIRC help channel
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Russel's loop worked as he wrote it. I tested it. ???

Originally Posted By: RusselB


Code:
//var %a = 1, %b = $hget(<table>,0).item | while %a <= %b { echo -a $hget(<table>,%a).item is $hget(<table>,%a) | inc %a }






I see what you are saying about the '1' as the first item in the field for the host, which is the way a table works:

1.2.3.4 1

on a /hget for 1.2.3.4 /hget would return a 'true' thus letting the 'if' continue.


I registered; you should too.
Joined: Jul 2006
Posts: 3,971
W
Hoopy frood
Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,971
I don't said that his code doesn't work, i just said the $hget value will be $null with your exemple


#mircscripting @ irc.swiftirc.net == the best mIRC help channel
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Originally Posted By: Wims
I don't said that his code doesn't work, i just said the $hget value will be $null with your exemple


Ah.


I registered; you should too.
Joined: Aug 2004
Posts: 7,252
R
Hoopy frood
Offline
Hoopy frood
R
Joined: Aug 2004
Posts: 7,252
Quote:
Thanks. I just wanted to verify the 6 hosts in the table. They are there.

Do you have any idea why the $hfind works for me, but the $hget doesn't?
Post the two codes that you tried.. the $hfind that worked and the $hget that didn't. I'd be willing to bet that you're not supplying $hget with the correct information to get the proper return, but by comparing the two codes, I should be able to tell and, in turn, tell you what you're doing wrong.

Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Originally Posted By: RusselB
[quote]Post the two codes that you tried.. the $hfind that worked and the $hget that didn't. I'd be willing to bet that you're not supplying $hget with the correct information to get the proper return, but by comparing the two codes, I should be able to tell and, in turn, tell you what you're doing wrong.


$2 = 1.2.3.4

Works: if ($hfind(bots,$2,1)) {

Doesn't work: if ($hget(bots,$2)) {

The Bot table is:

bots 1.2.3.4
bots 5.6.7.8
bots 2.3.4.5
bots 6.7.8.9
bots 3.4.5.6
bots 7.8.9.0


I registered; you should too.
Joined: Oct 2003
Posts: 3,918
A
Hoopy frood
Offline
Hoopy frood
A
Joined: Oct 2003
Posts: 3,918
Then $2 isnt the key

You'd want to fix your hash table to set whatever $2 is as the key so that you can retrieve the data quicker.

Looking at your original posts it seems the HOST is $2, so you'd want to add THAT as the key, not the nickname.


- argv[0] on EFnet #mIRC
- "Life is a pointer to an integer without a cast"
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Originally Posted By: argv0
Then $2 isnt the key

You'd want to fix your hash table to set whatever $2 is as the key so that you can retrieve the data quicker.

Looking at your original posts it seems the HOST is $2, so you'd want to add THAT as the key, not the nickname.


the variables that get passed in are %nick %host %chan so wouldn't $2 be correct as the key? That's the way I had it.

if ($hget(bots,$2)) {

???


I registered; you should too.
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Okay here is what I did:

ITEM | DATA
---------------
1.2.3.4 1
5.6.7.8 1
2.3.4.5 1
3.4.5.6 1

So now when I type this on the command line:
//echo -s $hget(bots,2.3.4.5)
I get this result:
1
Which should let the 'if' execute.



I registered; you should too.
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
This works and I have a better understanding of hash files now. Originally I thought an item could be used as it's data (i.e. just the item without it's data). Is that even possible with a hash table? So, now if I want to use a hash table for just userhost, there needs to be something in the data field, like the screen name. Right? I.E.

Item | DATA
-------------------------------------------------
screenname1 screenname[12345678_12345678@1.2.3.4]

Anyone have any ideas for a simple lil' script to convert a HUGE text file to a hash table/file?



I registered; you should too.
Joined: Oct 2003
Posts: 3,918
A
Hoopy frood
Offline
Hoopy frood
A
Joined: Oct 2003
Posts: 3,918
"without data" would mean $null for the data value, but you might as well put a value to make it easier to $hget like you did above. 1 is a good choice, as is something like $true. Of course, you could also put the nickname as the data field, that way you could do reverse lookups (reverse lookups are done with $hfind)... this is probably required by your script later on anyway.

Converting a text file depends on what your text file format is like... since text is just text-- you need to define some format to be able to parse it into item / data.


- argv[0] on EFnet #mIRC
- "Life is a pointer to an integer without a cast"
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
In the text file each line has a userhost:

screenname[12345678_12345678@1.2.3.4]

Convert to hash table format:

screenname screenname[12345678_12345678@1.2.3.4]

This way I can search the hash table by screenname (item) to get all the userhosts (data) that use that screenname.

Though I'm a lil' bit confused as to how to use the host from each of those screenname's (item) userhost (data) and search the hash table for all the screennames that use that host. Like a reverse lookup.


I registered; you should too.
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
In a hash table, how do you keep the data 'in order' so that it's saved to each item 'uniformly?' Can you you write to specific keys (?) for an item? I.E.

ITEM.......|1........2....3.....4........5........6........DATA
---------------------------------------------------------------
screenname1 userhost time chan last_said userhost userhost ...
screenname2 userhost .... .... ......... userhost ........ ...
screenname3 userhost time chan last_said ........ ........ ...

Does this look right?


I registered; you should too.
Joined: Aug 2004
Posts: 7,252
R
Hoopy frood
Offline
Hoopy frood
R
Joined: Aug 2004
Posts: 7,252
As long as all of the same parameters are passed to the table, they will be stored in the order that you pass them.

Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Can someone clearify $hget for me? Lets call the hash table info. The uniformity of each item is:

screenname1 userhost time channel last_said

Then if: set %a screenname1

So would $get(info,a%) return 'userhost' or 'userhost time channel last_said'

To retrieve a particular key of an item, like time for screenname1, $get(info,a%,2).item

???


I registered; you should too.
Joined: Aug 2007
Posts: 334
Pan-dimensional mouse
Offline
Pan-dimensional mouse
Joined: Aug 2007
Posts: 334
$hget returns everthing on an item

so:
example:
hadd Table item test item name blah blah blah
$hget(Table,Item)
returns: test item name blah blah blah


This is not the signature you are looking for
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Thanks.

I have a 10,000+ text file with userhostmasks. I need help with reading it into a hash table called iptracker. I understand how to /hmake -s iptracker 1000 | /hadd -s iptracker $nick[12345678_12345678@1.2.3.4] | hname/hsave -so text iptracker.hsh | /hload -so iptracker iptracker.hsh | /hfree iptracker | etc.

Each line in the text file is in the format:
screenname[12345678_12345678@1.2.3.4


/hmake -s iptracker 1000
/hsave -so iptracker iptracker.hsh

//var %a = 1, %b = $read(iptracker.txt,1) | echo -s %a %b | while %a <= %b { var %c = $read(iptracker.txt,%a) | echo -s %c | var %nick = $gettok(%c,1,91) | /hadd -s iptracker %nick %c | inc %a | unset %nick %c }

This works... but is there a better way?

RusselB's example to read a hash file:

//var %a = 1, %b = $hget(<table>,0).item | while %a <= %b { echo -a $hget(<table>,%a).item is $hget(<table>,%a) | inc %a }

Fixed:

//var %a = 1, %b = $hget(<table>,0).item | while %a <= %b { echo -a $hget(<table>,%a).item is $hget(<table>,%a).data | inc %a }


I registered; you should too.
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Originally Posted By: foshizzle
$hget returns everthing on an item

so:
example:
hadd Table item test item name blah blah blah
$hget(Table,Item)
returns: test item name blah blah blah


iptracker
screenname screenname[12345678_12345678@1234]

$hget(iptracker)
returns: iptracker

%hget(iptracker,screenname)
returns: screenname[12345678_12345678@1.2.3.4]

%hget(iptracker,1).item
returns: screenname[12345678_12345678@1.2.3.4]


I registered; you should too.
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
On second thought I'm not so sure it's a good idea to strip the nick off the user userhostmask and use that for the item. I'm reconsidering just using the whole userhostmask as the item, but then what would I use for the data part?


I registered; you should too.
Joined: Oct 2005
Posts: 1,741
G
Hoopy frood
Offline
Hoopy frood
G
Joined: Oct 2005
Posts: 1,741
You can use anything for the data. For example: the $ctime of when the data was added, or just the nick, or simply a 1 to indicate TRUE.

-genius_at_work

Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
I guess I could always change the format later on, if needed


I registered; you should too.
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
How can I prevent the following from showing an 'insufficant parameters' for /hadd after the last text file line read:

Code:
alias iptracker.init {
  var %a = 1, %b = $read(iptracker.txt,%a)
  while (%a <= %b) {
    var %b = $read(iptracker.txt,%a)
    var %nick = $gettok(%b,1,91)
    echo -s a: %a  nick: %nick  b: %b
    /hadd -s iptracker %nick %b
    inc %a
  }
}


I registered; you should too.
Joined: Jul 2006
Posts: 3,971
W
Hoopy frood
Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,971
Quote:
var %a = 1, %b = $read(iptracker.txt,%a)
sould be
Quote:
var %a = 1, %b = $read(iptracker.txt,0)


#mircscripting @ irc.swiftirc.net == the best mIRC help channel
Joined: May 2008
Posts: 329
A
AWEstun Offline OP
Fjord artisan
OP Offline
Fjord artisan
A
Joined: May 2008
Posts: 329
Then the while look won't execute as %a will be 1 and %b will be $null.

What I ended up doing was preventing the /hadd if %b = $null

if (%b != $null) /hadd ...

Code:
alias iptracker.init {
  var %a = 1, %b = 1
  while (%a = %b) {
    var %c = $read(iptracker.txt,%a)
    var %nick = $gettok(%c,1,91)
    echo -s a: %a  nick: %nick  c: %c
    if (%c != $null) {
      /hadd -s iptracker %nick %b
      inc %a
    }
    var %b = $calc($readn + 1)
  }
}


I registered; you should too.
Joined: Aug 2004
Posts: 7,252
R
Hoopy frood
Offline
Hoopy frood
R
Joined: Aug 2004
Posts: 7,252
Code:
alias iptracker.init {
  var %a = 1, %b = $lines(iptracker.txt)
  while (%a <= %b) {
    var %b = $read(iptracker.txt,%a)
    var %nick = $gettok(%b,1,91)
    echo -s a: %a  nick: %nick  b: %b
    /hadd -s iptracker %nick %b
    inc %a
  }
}


Joined: Jul 2006
Posts: 3,971
W
Hoopy frood
Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,971
Yes, i was tired, use RusselB's code

Last edited by Wims; 11/05/08 12:52 PM.

#mircscripting @ irc.swiftirc.net == the best mIRC help channel
Page 1 of 3 1 2 3

Link Copied to Clipboard