mIRC Home    About    Download    Register    News    Help

Print Thread
Stripping/Reading Files #110779
10/02/05 06:27 PM
10/02/05 06:27 PM
Joined: Dec 2004
Posts: 81
Maryland
N
nycdiesel Offline OP
Babel fish
nycdiesel  Offline OP
Babel fish
N

Joined: Dec 2004
Posts: 81
Maryland
I'm working on a script which downloads the search results web page from a webserver. I know how I can do this but was just curious if there's a better way. I'm trying to workaround the /set: line too long error in mIRC when sockreading. So I've decided to write the file to disk and then parse the file though I'm still getting errors. The problem is all of the TAB chrs at the beginning of lines in the file.
Code:
		<tr>


Any ideas?

Re: Stripping/Reading Files #110780
10/02/05 07:10 PM
10/02/05 07:10 PM
Joined: Sep 2004
Posts: 237
New Orleans
JAFO Offline
Fjord artisan
JAFO  Offline
Fjord artisan

Joined: Sep 2004
Posts: 237
New Orleans
Nice Code! Come up with that all on your own? wink
Perhaps its not too late to edit that and add the code.

Re: Stripping/Reading Files #110781
10/02/05 08:45 PM
10/02/05 08:45 PM
Joined: Dec 2004
Posts: 81
Maryland
N
nycdiesel Offline OP
Babel fish
nycdiesel  Offline OP
Babel fish
N

Joined: Dec 2004
Posts: 81
Maryland
Actually figured it out smile

Code:
alias test2 {
  var %idx = 0, %b, %chr
  write -c txt.txt
  fopen a page.txt
  while (!feof) {
    if ($ferr || $feof) { fclose a | return }
    %b = $fread(a,4096,&var)
    if (%b) {
      while (%idx < %b) {
        inc %idx
        %chr = $bvar(&var,%idx)
        if ($istok(10.13,%chr,46)) { bset &new %idx %chr }
        elseif (!$istok(10.13.9,%chr,46)) { bset &new %idx %chr }
      }
      bwrite txt.txt -1 &new
      %idx = 0
      bunset &new
    }
  }
}


It's still a little slow and freezes mIRC for about 15-30 seconds, not too bad, I suppose.

Last edited by nycdiesel; 10/02/05 09:40 PM.
Re: Stripping/Reading Files #110782
10/02/05 11:26 PM
10/02/05 11:26 PM
Joined: Sep 2003
Posts: 4,230
D
DaveC Offline
Hoopy frood
DaveC  Offline
Hoopy frood
D

Joined: Sep 2003
Posts: 4,230
Im supprised this was all that was needed to reduce lines to managable sizes, but if it is it is.

Use this its alot faster.

Code:
alias test2 {
  var %idx = 0, %b, %chr
  write -c txt.txt
  .fopen a page.txt
  while (!feof) {
    if ($ferr || $feof) { .fclose a | return }
    %b = $fread(a,4096,&var)
    if (%b) {
      if (1) {
        while ($bfind(&var, $v1, 9)) {
          bset &var $v1 0
        }
        bwrite txt.txt -1 &var
      }
    }
  }
}


The if (1) { is needed to preload $v1 for the while loop

Re: Stripping/Reading Files #110783
11/02/05 02:36 AM
11/02/05 02:36 AM
Joined: Dec 2004
Posts: 81
Maryland
N
nycdiesel Offline OP
Babel fish
nycdiesel  Offline OP
Babel fish
N

Joined: Dec 2004
Posts: 81
Maryland
Thanks Dave, much appreciated smile I wasn't too familiar with those v1, v2 things but I did my homework, heh. Anyways I'm still encountering a problem when attempting to parse the file after calling test2 .. *shrugs*

my second test alias echoed the contents of the file and it to the end of the file but it was only showing partial lines.

Re: Stripping/Reading Files #110784
11/02/05 06:10 AM
11/02/05 06:10 AM
Joined: Sep 2003
Posts: 4,230
D
DaveC Offline
Hoopy frood
DaveC  Offline
Hoopy frood
D

Joined: Sep 2003
Posts: 4,230
try this,

Code:
alias test1 {
  var %idx = 0, %b, %chr
  write -c txt.txt
  .fopen a page.txt
  while (!feof) {
    if ($ferr || $feof) { .fclose a | return }
    %b = $fread(a,4096,&var)
    if (%b) {
      if (1) {
        while ($bfind(&var, $v1, 9)) {
          bset &var $v1 10
        }
      }
      if (1) {
        while ($bfind(&var, $v1, 60)) {
          bset &var $v1 10
        }
      }
      bwrite txt.txt -1 &var
    }
  }
}


I dumped the 0 byte as i remebered it terminates the line, following data well not read from that line.

I now change TAB to LF and < to LF, this should be enough to break the lines down to managable sizes.

Just remeber that you cant search for <TR> sine it well just be TR> at the start of a line, and also that there well be $null length lines in the file.

Re: Stripping/Reading Files #110785
11/02/05 11:44 AM
11/02/05 11:44 AM
Joined: Aug 2003
Posts: 1,831
I
Iori Offline
Hoopy frood
Iori  Offline
Hoopy frood
I

Joined: Aug 2003
Posts: 1,831
This writes it back to the same file, if you really want the extra file just change the blue page.txt
Code:
alias testb {
  bread page.txt 1 $file(page.txt) &amp;a
  breplace &amp;a 9 10 60 10
  bwrite [color:blue]page.txt[/color] 1 -1 &amp;a
}

Re: Stripping/Reading Files #110786
11/02/05 11:54 AM
11/02/05 11:54 AM
Joined: Aug 2003
Posts: 1,831
I
Iori Offline
Hoopy frood
Iori  Offline
Hoopy frood
I

Joined: Aug 2003
Posts: 1,831
That last post of mine should have been directed to you.
I noticed this in your code
  • while (!feof) {

Should be
  • while (!$feof) {
they way you have it there "!feof" is just a string (which would always be true), same as Daves "if (1)"

  • //linesep | if !feof { echo -a $v1 } | if feof { echo -a $v1 }

Re: Stripping/Reading Files #110787
11/02/05 08:50 PM
11/02/05 08:50 PM
Joined: Sep 2003
Posts: 4,230
D
DaveC Offline
Hoopy frood
DaveC  Offline
Hoopy frood
D

Joined: Sep 2003
Posts: 4,230
Quote:
This writes it back to the same file, if you really want the extra file just change the blue page.txt
Code:
alias testb {
  bread page.txt 1 $file(page.txt) &amp;a
  breplace &amp;a 9 10 60 10
  bwrite [color:blue]page.txt[/color] 1 -1 &amp;a
}


Oh that is way nicer method, I was just adjusting his code, since i saw the byte by byte checking, and thought Im sure u can just find the values u want in the var instead, should have stepped back and looked at what he wanted as a result, rather than how to improve his code. Goes the same for the while (!feof) { I didnt even look at it frown