mIRC Home    About    Download    Register    News    Help

Print Thread
[Sockets] efficient tcp<->file tcp<->tcp copying #134270 30/10/05 01:01 PM
Joined: Oct 2005
Posts: 17
B
baa Offline OP
Pikka bird
OP Offline
Pikka bird
B
Joined: Oct 2005
Posts: 17
short desc purpose: very simple, copy between tcp<->file, tcp<->tcp
long desc purpose: 1 make a proxy in script works in a special way, and have someting else to use it, mostly mirc server/dcc connection, 2. faster(much less cpu) recving blocks(Nbytes) of binary data to file
tcp<->file problem: it's simple with current /sock* $sock*, but it's slow
tcp<->tcp problem: it's hard to deal when source tcp socket is recving too much data to push into the target socket.
don't /sockread source socket if target is full? noway, mirc think u're giving up the source socket.
sockread 1 (or 0 if possible) byte? no, mirc re-trigger on sockread like crazy.
buffer to file? it's random, not FIFO, and all data is stored in hd, takes up space if u transfer 100MB or so. you can switch to new buffer file when it reach 1mb and delete the old correctly, a bit complex to implement. and it's dumb if you're copying from local-end socket to remote-end, all data from localsocket is so fast that bump the buffer file.

problem in short: 2 socket can't be made "relatived"
let's go for it
Code:
on 1:sockread:source: {
  var %buffer
  :nextread
  sockread source %buffer
  if (!$sockbr) {
    return
  }

  parse %buffer
  var %bytes = $gettok(%buffer, ....)
  if (decided to copy) {
    sockcopy %bytes source target
    ; or u can do:
    ; -t tcp to file
    ; -f from file to tcp
    ; sockcopy -t %bytes source targetfile

    ; sockcopy just setup a copy mission, does NOT wait until the copy is done
    ; so return right now
    ; sockread will never triggered from now until %bytes of data is copied
    return
  }
  do something else

  goto nextread
}
on 1:sockcopy:source: {
  ; trigger when copy is done
  ; do something here if u wan't
  ; sockread is triggered after return
  return
}


$sock(source/N).sourceleft
return how much bytes left to finish the /sockcopy FROM this socket

$sock(target/N).targetleft
return how much bytes left to finish the /sockcopy TO this socket.
$sock(target/N).targetleft == $sock(source/N).sourceleft if u're copying tcp->tcp, but this is useful if u copy from file->socket

the mirc engine will do all the copy mission, taking care if one of the socket is blocked and marking the other socket "wait"
there's no relation between "/sockcopy a b" and "/sockcopy b a"

well, this is not new feature in mirc document. but a feature request

if this is "nice" feature is too complex to implmenent, there's another way:

/sockread -s [sockname]
; stop re-triggering socket event, but mirc can read the socket to its socket buffer

/sockread -c <sockname>
; continue a socket read trigger
yet cannot speed up tcp<->file

Last edited by baa; 30/10/05 01:12 PM.
Re: [Sockets] efficient tcp<->file tcp<->tcp copyi #134271 30/10/05 04:00 PM
Joined: Apr 2003
Posts: 701
K
Kelder Offline
Hoopy frood
Offline
Hoopy frood
K
Joined: Apr 2003
Posts: 701
How can the data from a socket be random and not FIFO? I have never seen random-access sockets. It can be that the data sent over the socket contains any kind of stuff, but that can't matter if you want to forward the contents of it without parsing it...

A possible workaround for your buffering problems is a hashtable containing binary variables. This prevents disk access, but can fill your memory it you have to buffer large amounts of data. The file option is useable too, it's not because you send a xxx gigabyte file that you need to assign that much space. Also imho such large files might be best handled by a 'real' scripting language or programming language.

Re: [Sockets] efficient tcp<->file tcp<->tcp copyi #134272 30/10/05 05:51 PM
Joined: Apr 2004
Posts: 862
Sat Offline
Hoopy frood
Offline
Hoopy frood
Joined: Apr 2004
Posts: 862
stream -> file: slow? don't forget you're using a scripting language..

file -> stream: some have suggested before that "on SOCKWRITE" should be triggered whenever mIRC has sent some of the data, instead of all of it. That would take care of the problem here.

stream -> stream: it would be nice if mIRC used its own buffering for outgoing data, instead of relying on the fixed-size buffer that winsock provides.

The things you suggest are way too specific for my liking at least..


Saturn, QuakeNet staff
Re: [Sockets] efficient tcp<->file tcp<->tcp copyi #134273 31/10/05 06:40 AM
Joined: Oct 2005
Posts: 17
B
baa Offline OP
Pikka bird
OP Offline
Pikka bird
B
Joined: Oct 2005
Posts: 17
no, i never expect a random access tcp, but just copy tcp<->tcp tcp<->file(append)
they should blocks each other on either way

Quote:
stream -> file: slow? don't forget you're using a scripting language..

i know that smile
never expect a too fast socketing in script

Quote:
file -> stream: some have suggested before that "on SOCKWRITE" should be triggered whenever mIRC has sent some of the data, instead of all of it. That would take care of the problem here.

thanks for telling me about this. it's slow yet i can live with it.

Quote:
stream -> stream: it would be nice if mIRC used its own buffering for outgoing data, instead of relying on the fixed-size buffer that winsock provides.


nice suggestions. i can still do nothing if the source is sending me too fast and the target is write-queue-full, until my suggestion is applied.