mIRC Home    About    Download    Register    News    Help

Print Thread
#266440 09/12/19 12:06 PM
Joined: Jun 2017
Posts: 9
M
Nutrimatic drinks dispenser
OP Offline
Nutrimatic drinks dispenser
M
Joined: Jun 2017
Posts: 9
Hey There,

I would need some help with my textfile processing.

Following situation:
Textfile with around 20.000 lines.
(every line contains a msg message)

My script is Processing as follows:

1) read the first line from txt (with read command)
2) delete the first line from txt (with write command)
3) if the txt file is not empty start again with 1) (in a loop)
4) if the txt file is empty - halt

Issue:
I think the issue which i provide as follows comes from the txt file size.

At the beginning of the processing the script only reads 3-5 lines per second.
Later it is getting faster and faster up to 20-40 lines per second.

Is there a way to process large text files faster?

Joined: Jul 2006
Posts: 4,146
W
Hoopy frood
Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 4,146
they are better ways to read a text file but it would depends on what you need, if you know you want to process all the line or most of them, you can use /filter, if you want some specific line(s) but only a few, File handling can be used.
Seems like you're processing all the lines, so you should check /filter: https://en.wikichip.org/wiki/mirc/commands/filter


#mircscripting @ irc.swiftirc.net == the best mIRC help channel
Joined: Dec 2015
Posts: 148
Vogon poet
Offline
Vogon poet
Joined: Dec 2015
Posts: 148
You could just use /filter, and then /write -c to clear the file.

As an example: //filter -fk mymessages.txt myalias | write -c mymessages.txt

Or if you just want to test WITHOUT CLEARING THE FILE, try: /filter -fk mymessages.txt myalias
Code
alias myalias {
  ;$1 contains the whole line.
  echo -s Here's the whole line: $1

  ;This thing here splits the input line into tokens separated by character 32, which is a normal empty space.
  tokenize 32 $1
  echo -s There are $0 words! First word: $1 Second word: $2 Third word: $3 $+ ...
}


/help /filter
/help /write

Joined: Jun 2017
Posts: 9
M
Nutrimatic drinks dispenser
OP Offline
Nutrimatic drinks dispenser
M
Joined: Jun 2017
Posts: 9
Great that is what I was looking for - now I have another issue - is there a way to limit the output to 12 messages per second?

Joined: Jan 2004
Posts: 2,127
Hoopy frood
Offline
Hoopy frood
Joined: Jan 2004
Posts: 2,127
/timername -m 0 $calc(1000//12) alias_that_does_one_line

or you can have a timer that functions once per second and processes a burst of 12.
You could also have all the lines processed to another disk file using the complete output, then /play it with interval of $calc(1000//12) milliseconds.

Joined: Dec 2015
Posts: 148
Vogon poet
Offline
Vogon poet
Joined: Dec 2015
Posts: 148
Just to throw in one more option:
Code
alias mytimer {
  set -e %myvariable 1
  .timermytimer -o 0 1 myfilter
}

alias -l myfilter {
  filter -fkr $+(%myvariable,-,$calc(11+%myvariable)) mymessages.txt myalias
  if ($filtered) inc %myvariable 12
  else {
    unset %myvariable
    .timermytimer off
    write -c mymessages.txt
    echo -a Done!
  }
}

alias -l myalias {
  echo -s Here's the whole line: $1
}

/mytimer starts a timer that spits out 12 lines at a time, and clears the file after there's no more lines left.

If you stop it early and start again, it will repeat the same lines.


Link Copied to Clipboard