hi bleach! a lot of $read() overuse there, coupled with a bit of a "naive" algorithm, and i can totally understand the slow running time laugh

this method should prove to be substantially faster:

Code:
alias d.detect {
  var %file = $+(",$sfile(*.txt,select file...,start),")
  var %o.file = $replace(%file,.txt",-ok.txt")
  .fopen d.in %file
  if (!$ferr) {
    .fopen -o d.out %o.file
    if (!$ferr) {
      if ($hget(d.detect)) hfree d.detect
      hmake d.detect $iif($calc($lines(%file) / 10 + 1) > 10000,10000,$v1) 
      while (!$fopen(d.in).eof) {
        var %line = $encode($fread(d.in) $+ $crlf,m)
        if (!$hget(d.detect,%line)) .fwrite d.out $decode(%line,m)
        hinc d.detect %line
      }
    }
    .fclose d.out
  }
  .fclose d.in
}


that should run in time proportional to the size of the file, whereas your current alias runs several orders of magnitude slower

"The only excuse for making a useless script is that one admires it intensely" - Oscar Wilde