Updating file via socket instead of overwrite.

Well that's about as specific as I can make the topic title.
I'm not sure if it's possible, but I currently have a socket script that downloads a .txt file from a server, but everytime I want to update the file I have to rewrite it completely. As it has been becoming quite large lately (almost 800kB) the download is getting slower and slower.

Now I'd like to fix this before it actually starts to cause problems. Is there a possibility to check where the file on the server changed compared to the file on my hard drive and update it accordingly?

There are lines added, modified and deleted over the entire file every hour, it doesn't just add new lines to the end.

Well I hope this is kind of clear, if you read it a few times it should make sense. Now let me hope it's possible.

Use the If-Modified-Since header. An example follows.

Code:

; Type /upd to begin.
 
alias upd {
  set %upd.site    www.example.com
  set %upd.request /
  set %upd.file    example.html
  set %upd.header  1
  sockclose upd
  sockopen upd %upd.site 80
  _info Connecting to %upd.site
}
 
alias -l _info echo -sc info * upd: $1-
alias -l _cleanup {
  sockclose upd
  unset %upd.*
}
 
On *:sockopen:upd:{
  if $sockerr {
    _info Error: unable to connect.
    _cleanup
    return
  }
  _info Requesting file %upd.request
  sockwrite -tn $sockname GET %upd.request HTTP/1.1
  sockwrite -tn $sockname Host: %upd.site
  sockwrite -tn $sockname Connection: close
  if $isfile(%upd.file) {
    sockwrite -tn $sockname If-Modified-Since: $&amp;
      $gmt($file(%upd.file).mtime)
  }
  sockwrite -tn $sockname 
}
 
On *:sockread:upd:{
  if %upd.header {
    var %var
    sockread %var
    tokenize 32 %var
    if HTTP/* iswm $1 {
      if     $2 == 200 { 
        ; OK
        _info Receiving data...
        .remove %upd.file
      }
      elseif $2 == 304 {
        ; Not Modified
        _info File not modified.
      }
      else {
        _info Error: unrecognized reply: $1-
        _cleanup
      }
    }
    elseif !$1 {
      ; A blank line ends the header section
      set %upd.header 0
    }
  }
  else {
    _info Writing to disk: %upd.file
    sockread &amp;var
    bwrite %upd.file -1 &amp;var
  }
}
 
On *:sockclose:upd:{
  _info Done.
  _cleanup
}

Thanks, but wouldn't that just download the entire file again if it has changed since that certain time?

I know it changes every hour, but I'm searching a way so I only have to download the lines that were changed or added.

And find a way to delete the lines that don't exist in the file on the server anymore.

That is not possable, since you cant findout the modified lines untell you have the new file to compare to the old file, then you already have downloaded the new file, so why compare them. Downloading if the file is newer is the only method you have, unless the website was to develop a seperate updates only file, which i highly doubt would ever happen.

If the file being downloaded is only updated at the end of the file then a resume (range) can be done instead of a full download.

Do you even read things before you reply to them?

Quote:

There are lines added, modified and deleted over the entire file every hour, it doesn't just add new lines to the end.

Quote:

That is not possable, since you cant findout the modified lines untell you have the new file to compare to the old file, then you already have downloaded the new file, so why compare them.

That's what I thought, thanks for the help anyway.