Code:
alias undupe {
  .fopen -no tmp unduped.txt
  filter -fk $qt($1) undupe_wf *
  .fclose tmp
}

alias -l undupe_wf .fwrite -n tmp $token($1-,2-,32)


Code goes in remotes (alt + r)

Useage: /undupe <filename>

After execution, a "clean" version of the file will be stored in unduped.txt inside your mIRC directory.

As per the previous solution given, the format of the lines is assumed to be consistent throughout the file, else this won't work as expected. This is also much faster than the previous solutions, especially on files with a large number of lines (thousands).