mIRC Home    About    Download    Register    News    Help

Print Thread
Sockets issues #237413 02/05/12 01:29 PM
Joined: Feb 2003
Posts: 307
T
tontito Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Feb 2003
Posts: 307
Hello to you all,

There are some things that seem to me like a bug but might be miss interpretation, so I am posting here just in case

/socklisten command doesn't start successfully if there are no other sockets listening on that port, but there are sockets attempting to connect or old sockets in time_wait status (meaning they are closing the client connection)

/sockpause doesn't seem to pause the way I expected, if I am receiving data and I want to signal the sender to stop for a bit, it doesn't, I keep receiving sockread events.
In this case, if I have 2 sockets in cascade (aka proxy) I don't have many ways to avoid losing data but to write the extra information in file or memory until the "client" free some space in send buffer (I don't want to exceed the max of 16k bytes).

Are these really bugs?

Regards

Re: Sockets issues [Re: tontito] #237431 03/05/12 12:10 PM
Joined: Jul 2006
Posts: 3,810
W
Wims Offline
Hoopy frood
Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,810
Quote:
/socklisten command doesn't start successfully if there are no other sockets listening on that port, but there are sockets attempting to connect or old sockets in time_wait status (meaning they are closing the client connection)
I'm not sure about what you meant to say, but I can confirm that under some circumstances, mIRC used to report a /socklisten port error whereas all sockets were closed; I'm expecting /socklisten to work again without waiting a little bit.

Btw, there's a real bug in /socklisten, it fails to report a bind() error if you listen to the same port:
Code:
//socklisten test 8000 | socklisten test1 8000




Looking for a good help channel about mIRC? Check #mircscripting @ irc.swiftirc.net
Re: Sockets issues [Re: tontito] #237441 04/05/12 10:24 PM
Joined: Feb 2003
Posts: 307
T
tontito Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Feb 2003
Posts: 307
I have made some more tests and I know why I was thinking sockpause wasn't working.

Basically I found that receive queue is a bigger then 16k bytes.

I am not sure what is the standard for common sockets (on other applications) but I think it would be best to have same size for both, or be able to set the size of the receiving.

If between two sockreads I get 20k bytes (this is a true scenario) in the queue, the sockpause is totally useless by then.

Regards

Re: Sockets issues [Re: tontito] #237443 05/05/12 12:01 AM
Joined: Apr 2010
Posts: 966
F
FroggieDaFrog Offline
Hoopy frood
Offline
Hoopy frood
F
Joined: Apr 2010
Posts: 966
To read everything in the read buffer use something along the lines of:
Code:
sockread $sock($sockname).rq &bvar


I am SReject
My Stuff
Re: Sockets issues [Re: FroggieDaFrog] #237446 05/05/12 02:40 PM
Joined: Feb 2003
Posts: 307
T
tontito Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Feb 2003
Posts: 307
Correct!

Now imagine you need to write that on another socket that has it's sq is more then half full.

I was trying to make this work without having to store data in a hashtable until the other socket sq is a little bit more empty.

Regards

Re: Sockets issues [Re: tontito] #237447 05/05/12 03:06 PM
Joined: Dec 2002
Posts: 344
D
drum Offline
Pan-dimensional mouse
Offline
Pan-dimensional mouse
D
Joined: Dec 2002
Posts: 344
Originally Posted By: tontito
Correct!

Now imagine you need to write that on another socket that has it's sq is more then half full.

I was trying to make this work without having to store data in a hashtable until the other socket sq is a little bit more empty.

Regards


Even if the send and receive queues are the exact same size, you will have to deal with this issue because you can't guarantee there will be enough room in the send queue for the amount of data you are pulling from the receive queue.

In other words, if you don't use a hash table when data overflows, you will lose data.

Re: Sockets issues [Re: drum] #237448 05/05/12 03:18 PM
Joined: Feb 2003
Posts: 307
T
tontito Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Feb 2003
Posts: 307
Thanks for you feedback and with what i am finding, I can't agree more with you.

Anyway, I was hoping to use sockpause before I get that.

Since the receive buffer was a lot smaller, any big about of data (the 20k at once) would stop at middle, making the tcp entering in congestion avoidance.

In resume, have a better way to fine-tune socket handling.

Btw, have any of you used sockpause then sockpause -r (this is also self set when the buffer gets 0 bytes)?

From my experience after I resume it the data transfer isn't recovered.
I believe I am missing some other thing.

Re: Sockets issues [Re: tontito] #237450 05/05/12 06:08 PM
Joined: Jul 2006
Posts: 3,810
W
Wims Offline
Hoopy frood
Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,810
Quote:
/sockpause doesn't seem to pause the way I expected, if I am receiving data and I want to signal the sender to stop for a bit, it doesn't, I keep receiving sockread events.
In this case, if I have 2 sockets in cascade (aka proxy) I don't have many ways to avoid losing data but to write the extra information in file or memory until the "client" free some space in send buffer (I don't want to exceed the max of 16k bytes).
Quote:
Basically I found that receive queue is a bigger then 16k bytes.


You cannot prevent a client from sending data, /sockpause only stop the on sockread event from triggering for that socket.
Pausing the socket doesn't stop mIRC from receiving data from the socket, when you resume the socket, the data is there, you don't lose anything using /sockpause
The receive buffer can be of any size in theory, it's not important, you are not limited by this but by the send buffer. /sockpause simply isn't what you need and works fine.
Note that if the send queue is empty, you can send more than 16384 bytes, might be a bug.


Looking for a good help channel about mIRC? Check #mircscripting @ irc.swiftirc.net
Re: Sockets issues [Re: Wims] #237451 05/05/12 06:30 PM
Joined: Feb 2003
Posts: 307
T
tontito Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Feb 2003
Posts: 307
Originally Posted By: Wims

Note that if the send queue is empty, you can send more than 16384 bytes, might be a bug.


Thanks for your clarifications, but I am not so sure about the last statement.

I think you have never seen this message

* /sockwrite: 'route_mysql_s_36' queue would exceed 16384 bytes

Anyway I finished reimplementing the routing stuff without the sockpause, for sure it isn't a command for me, the Hashtable is all I need to keep the sockets running at max speed.

Regards

Re: Sockets issues [Re: tontito] #237452 05/05/12 06:40 PM
Joined: Jul 2006
Posts: 3,810
W
Wims Offline
Hoopy frood
Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 3,810
I know about this message, re-read the statement, I said "if the send queue is empty":
Code:
alias serv_test { sockclose serv_test | socklisten -p serv_test $iif($1,$1,8000) }
on *:sockread:client_test?*:{
  sockread $sock($sockname).rq &serv_test
  if ($bvar(&serv_test,0)) echo -a $v1 bytes $bvar(&serv_test,1-4000).text
}
on *:socklisten:serv_test:sockaccept client_test $+ $calc($sock(client_test*,0) +1)
alias client_test { sockclose client_test | sockopen client_test 127.0.0.1 $iif($1,$1,8000) }
Use this, type "//serv_test | client_test" and then type "//bset &a 80000 0 | sockwrite client_test &a", now compare with "//bset &a 16384 0 | sockwrite client_test &a | sockwrite -n client_test"

Last edited by Wims; 05/05/12 07:00 PM.

Looking for a good help channel about mIRC? Check #mircscripting @ irc.swiftirc.net
Re: Sockets issues [Re: Wims] #237454 05/05/12 06:57 PM
Joined: Jan 2004
Posts: 1,328
L
Loki12583 Offline
Hoopy frood
Offline
Hoopy frood
L
Joined: Jan 2004
Posts: 1,328
Originally Posted By: Tontito
I keep receiving sockread events

Originally Posted By: Wims
/sockpause only stop the on sockread event from triggering for that socket.

There seems to be some confusion here. Even if /sockpause isn't what he needs (I think it would be if it did stop the sockread, because he could resume the sockreads when he's ready to send more data) - what does it do?

Re: Sockets issues [Re: Loki12583] #237455 05/05/12 07:09 PM
Joined: Dec 2002
Posts: 344
D
drum Offline
Pan-dimensional mouse
Offline
Pan-dimensional mouse
D
Joined: Dec 2002
Posts: 344
Originally Posted By: Wims
(I think it would be if it did stop the sockread, because he could resume the sockreads when he's ready to send more data)


It's not good because the receive buffer is not an infinite size, and it could fill more rapidly than the send buffer empties.

To illustrate this point in a basic way, suppose your bandwidth is capped at 50KB/sec upload (but your download speed is faster). You receive data at a steady 60KB/sec from socket A, and attempt to send that data to socket B, but your internet connection limits it to 50KB/sec. Within seconds you have a buffer overflow.

Re: Sockets issues [Re: drum] #237456 05/05/12 07:17 PM
Joined: Feb 2003
Posts: 307
T
tontito Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Feb 2003
Posts: 307
Guys, read this a bit

http://courses.washington.edu/ee461/handouts/TCP_Congestion_Avoidance.pdf

If the buffer is full the tcp layer itself will handle the data on the other applications, and will stop sending until the socket can receive more.

In our case we have control on that layer, that is why we need to keep checking how full it is.

This is my interpretation

Re: Sockets issues [Re: Wims] #237457 06/05/12 11:28 AM
Joined: Dec 2002
Posts: 5,002
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 5,002
Quote:
Btw, there's a real bug in /socklisten, it fails to report a bind() error if you listen to the same port

The latest versions of mIRC use the SO_REUSEADDR setting to mitigate issues relating to TCP_WAIT, which many users have reported over the years. This setting means that bind will not return an error when you attempt to listen on a port that is already in use, if that socket has SO_REUSEADDR already set.

Re: Sockets issues [Re: tontito] #237458 06/05/12 11:40 AM
Joined: Dec 2002
Posts: 5,002
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 5,002
Quote:
/socklisten command doesn't start successfully if there are no other sockets listening on that port, but there are sockets attempting to connect or old sockets in time_wait status

Even with the use of SO_REUSEADDR, mentioned in my previous post, there are situations where Windows will not allow you to bind to sockets that are in some states. There is no way around that as far as I can tell.

Quote:
/sockpause doesn't seem to pause the way I expected, if I am receiving data and I want to signal the sender to stop for a bit, it doesn't, I keep receiving sockread events.

This command does not communicate with the sender. Its purpose is to stop mIRC from reading data from Winsock. Winsock will buffer any incoming data until you decide to start reading again.

Re: Sockets issues [Re: Khaled] #237459 06/05/12 01:52 PM
Joined: Feb 2003
Posts: 307
T
tontito Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Feb 2003
Posts: 307
Hello Khaled, thank you for your feedback and clarifications.

Related to the socklisten, even when mirc can't bind a socket (after closing a listening socket), I am able to use it with any other application (as long there is no listen on that port using the netstat -an).

I have been working with sockets for several years with java and lately with .net, and the situations I found with this error were only when the port was actually binded (confirmed with the netstat).

I remember that before you made the first change (some years ago) it was actually working well.
By then you did the change to fix any particular error or was to update the socket handling?

Best regards

Re: Sockets issues [Re: tontito] #237461 06/05/12 06:17 PM
Joined: Dec 2002
Posts: 5,002
Khaled Offline
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 5,002
The TIME_WAIT issue has been reported many times over the years, across different versions of mIRC. There have been many changes to the socket routines in that time. TIME_WAIT prevents binding to a port for a few minutes after it has been closed. This is by design in Winsock and is there for a reason. It applies to all applications. In some cases, such as the use of SO_REUSEADDR, you may be able to get around it.

In current versions of mIRC, /socklisten should fail only if bind() fails. So far, when testing under XP, I have been unable to make /socklisten fail, however I am sure that there will be situations where it will fail, since there may be contexts in which Winsock will decide a bind is not allowed.

That said, if you can describe a step by step method of reproducing the issue, I will try it out.

Re: Sockets issues [Re: Khaled] #237500 09/05/12 09:54 PM
Joined: Feb 2003
Posts: 307
T
tontito Offline OP
Fjord artisan
OP Offline
Fjord artisan
T
Joined: Feb 2003
Posts: 307
Hi again,

Donno if you have tested it windows 7, but socklisten take considerable longer to start.

Otherwise works like in my Xp server.

Regards