|
Joined: Feb 2003
Posts: 306
Fjord artisan
|
OP
Fjord artisan
Joined: Feb 2003
Posts: 306 |
Hello to you all,
There are some things that seem to me like a bug but might be miss interpretation, so I am posting here just in case
/socklisten command doesn't start successfully if there are no other sockets listening on that port, but there are sockets attempting to connect or old sockets in time_wait status (meaning they are closing the client connection)
/sockpause doesn't seem to pause the way I expected, if I am receiving data and I want to signal the sender to stop for a bit, it doesn't, I keep receiving sockread events. In this case, if I have 2 sockets in cascade (aka proxy) I don't have many ways to avoid losing data but to write the extra information in file or memory until the "client" free some space in send buffer (I don't want to exceed the max of 16k bytes).
Are these really bugs?
Regards
|
|
|
|
Joined: Jul 2006
Posts: 4,020
Hoopy frood
|
Hoopy frood
Joined: Jul 2006
Posts: 4,020 |
/socklisten command doesn't start successfully if there are no other sockets listening on that port, but there are sockets attempting to connect or old sockets in time_wait status (meaning they are closing the client connection) I'm not sure about what you meant to say, but I can confirm that under some circumstances, mIRC used to report a /socklisten port error whereas all sockets were closed; I'm expecting /socklisten to work again without waiting a little bit. Btw, there's a real bug in /socklisten, it fails to report a bind() error if you listen to the same port: //socklisten test 8000 | socklisten test1 8000
#mircscripting @ irc.swiftirc.net == the best mIRC help channel
|
|
|
|
Joined: Feb 2003
Posts: 306
Fjord artisan
|
OP
Fjord artisan
Joined: Feb 2003
Posts: 306 |
I have made some more tests and I know why I was thinking sockpause wasn't working.
Basically I found that receive queue is a bigger then 16k bytes.
I am not sure what is the standard for common sockets (on other applications) but I think it would be best to have same size for both, or be able to set the size of the receiving.
If between two sockreads I get 20k bytes (this is a true scenario) in the queue, the sockpause is totally useless by then.
Regards
|
|
|
|
Joined: Apr 2010
Posts: 964
Hoopy frood
|
Hoopy frood
Joined: Apr 2010
Posts: 964 |
To read everything in the read buffer use something along the lines of: sockread $sock($sockname).rq &bvar
|
|
|
|
Joined: Feb 2003
Posts: 306
Fjord artisan
|
OP
Fjord artisan
Joined: Feb 2003
Posts: 306 |
Correct!
Now imagine you need to write that on another socket that has it's sq is more then half full.
I was trying to make this work without having to store data in a hashtable until the other socket sq is a little bit more empty.
Regards
|
|
|
|
Joined: Dec 2002
Posts: 294
Pan-dimensional mouse
|
Pan-dimensional mouse
Joined: Dec 2002
Posts: 294 |
Correct!
Now imagine you need to write that on another socket that has it's sq is more then half full.
I was trying to make this work without having to store data in a hashtable until the other socket sq is a little bit more empty.
Regards Even if the send and receive queues are the exact same size, you will have to deal with this issue because you can't guarantee there will be enough room in the send queue for the amount of data you are pulling from the receive queue. In other words, if you don't use a hash table when data overflows, you will lose data.
|
|
|
|
Joined: Feb 2003
Posts: 306
Fjord artisan
|
OP
Fjord artisan
Joined: Feb 2003
Posts: 306 |
Thanks for you feedback and with what i am finding, I can't agree more with you.
Anyway, I was hoping to use sockpause before I get that.
Since the receive buffer was a lot smaller, any big about of data (the 20k at once) would stop at middle, making the tcp entering in congestion avoidance.
In resume, have a better way to fine-tune socket handling.
Btw, have any of you used sockpause then sockpause -r (this is also self set when the buffer gets 0 bytes)?
From my experience after I resume it the data transfer isn't recovered. I believe I am missing some other thing.
|
|
|
|
Joined: Jul 2006
Posts: 4,020
Hoopy frood
|
Hoopy frood
Joined: Jul 2006
Posts: 4,020 |
/sockpause doesn't seem to pause the way I expected, if I am receiving data and I want to signal the sender to stop for a bit, it doesn't, I keep receiving sockread events. In this case, if I have 2 sockets in cascade (aka proxy) I don't have many ways to avoid losing data but to write the extra information in file or memory until the "client" free some space in send buffer (I don't want to exceed the max of 16k bytes). Basically I found that receive queue is a bigger then 16k bytes. You cannot prevent a client from sending data, /sockpause only stop the on sockread event from triggering for that socket. Pausing the socket doesn't stop mIRC from receiving data from the socket, when you resume the socket, the data is there, you don't lose anything using /sockpause The receive buffer can be of any size in theory, it's not important, you are not limited by this but by the send buffer. /sockpause simply isn't what you need and works fine. Note that if the send queue is empty, you can send more than 16384 bytes, might be a bug.
#mircscripting @ irc.swiftirc.net == the best mIRC help channel
|
|
|
|
Joined: Feb 2003
Posts: 306
Fjord artisan
|
OP
Fjord artisan
Joined: Feb 2003
Posts: 306 |
Note that if the send queue is empty, you can send more than 16384 bytes, might be a bug.
Thanks for your clarifications, but I am not so sure about the last statement. I think you have never seen this message * /sockwrite: 'route_mysql_s_36' queue would exceed 16384 bytes Anyway I finished reimplementing the routing stuff without the sockpause, for sure it isn't a command for me, the Hashtable is all I need to keep the sockets running at max speed. Regards
|
|
|
|
Joined: Jul 2006
Posts: 4,020
Hoopy frood
|
Hoopy frood
Joined: Jul 2006
Posts: 4,020 |
I know about this message, re-read the statement, I said "if the send queue is empty": alias serv_test { sockclose serv_test | socklisten -p serv_test $iif($1,$1,8000) }
on *:sockread:client_test?*:{
sockread $sock($sockname).rq &serv_test
if ($bvar(&serv_test,0)) echo -a $v1 bytes $bvar(&serv_test,1-4000).text
}
on *:socklisten:serv_test:sockaccept client_test $+ $calc($sock(client_test*,0) +1)
alias client_test { sockclose client_test | sockopen client_test 127.0.0.1 $iif($1,$1,8000) } Use this, type "//serv_test | client_test" and then type "//bset &a 80000 0 | sockwrite client_test &a", now compare with "//bset &a 16384 0 | sockwrite client_test &a | sockwrite -n client_test"
Last edited by Wims; 05/05/12 07:00 PM.
#mircscripting @ irc.swiftirc.net == the best mIRC help channel
|
|
|
|
Joined: Jan 2004
Posts: 1,330
Hoopy frood
|
Hoopy frood
Joined: Jan 2004
Posts: 1,330 |
I keep receiving sockread events /sockpause only stop the on sockread event from triggering for that socket. There seems to be some confusion here. Even if /sockpause isn't what he needs (I think it would be if it did stop the sockread, because he could resume the sockreads when he's ready to send more data) - what does it do?
|
|
|
|
Joined: Dec 2002
Posts: 294
Pan-dimensional mouse
|
Pan-dimensional mouse
Joined: Dec 2002
Posts: 294 |
(I think it would be if it did stop the sockread, because he could resume the sockreads when he's ready to send more data) It's not good because the receive buffer is not an infinite size, and it could fill more rapidly than the send buffer empties. To illustrate this point in a basic way, suppose your bandwidth is capped at 50KB/sec upload (but your download speed is faster). You receive data at a steady 60KB/sec from socket A, and attempt to send that data to socket B, but your internet connection limits it to 50KB/sec. Within seconds you have a buffer overflow.
|
|
|
|
Joined: Feb 2003
Posts: 306
Fjord artisan
|
OP
Fjord artisan
Joined: Feb 2003
Posts: 306 |
Guys, read this a bit http://courses.washington.edu/ee461/handouts/TCP_Congestion_Avoidance.pdfIf the buffer is full the tcp layer itself will handle the data on the other applications, and will stop sending until the socket can receive more. In our case we have control on that layer, that is why we need to keep checking how full it is. This is my interpretation
|
|
|
|
Joined: Dec 2002
Posts: 3,840
Hoopy frood
|
Hoopy frood
Joined: Dec 2002
Posts: 3,840 |
Btw, there's a real bug in /socklisten, it fails to report a bind() error if you listen to the same port The latest versions of mIRC use the SO_REUSEADDR setting to mitigate issues relating to TCP_WAIT, which many users have reported over the years. This setting means that bind will not return an error when you attempt to listen on a port that is already in use, if that socket has SO_REUSEADDR already set.
|
|
|
|
Joined: Dec 2002
Posts: 3,840
Hoopy frood
|
Hoopy frood
Joined: Dec 2002
Posts: 3,840 |
/socklisten command doesn't start successfully if there are no other sockets listening on that port, but there are sockets attempting to connect or old sockets in time_wait status Even with the use of SO_REUSEADDR, mentioned in my previous post, there are situations where Windows will not allow you to bind to sockets that are in some states. There is no way around that as far as I can tell. /sockpause doesn't seem to pause the way I expected, if I am receiving data and I want to signal the sender to stop for a bit, it doesn't, I keep receiving sockread events. This command does not communicate with the sender. Its purpose is to stop mIRC from reading data from Winsock. Winsock will buffer any incoming data until you decide to start reading again.
|
|
|
|
Joined: Feb 2003
Posts: 306
Fjord artisan
|
OP
Fjord artisan
Joined: Feb 2003
Posts: 306 |
Hello Khaled, thank you for your feedback and clarifications.
Related to the socklisten, even when mirc can't bind a socket (after closing a listening socket), I am able to use it with any other application (as long there is no listen on that port using the netstat -an).
I have been working with sockets for several years with java and lately with .net, and the situations I found with this error were only when the port was actually binded (confirmed with the netstat).
I remember that before you made the first change (some years ago) it was actually working well. By then you did the change to fix any particular error or was to update the socket handling?
Best regards
|
|
|
|
Joined: Dec 2002
Posts: 3,840
Hoopy frood
|
Hoopy frood
Joined: Dec 2002
Posts: 3,840 |
The TIME_WAIT issue has been reported many times over the years, across different versions of mIRC. There have been many changes to the socket routines in that time. TIME_WAIT prevents binding to a port for a few minutes after it has been closed. This is by design in Winsock and is there for a reason. It applies to all applications. In some cases, such as the use of SO_REUSEADDR, you may be able to get around it.
In current versions of mIRC, /socklisten should fail only if bind() fails. So far, when testing under XP, I have been unable to make /socklisten fail, however I am sure that there will be situations where it will fail, since there may be contexts in which Winsock will decide a bind is not allowed.
That said, if you can describe a step by step method of reproducing the issue, I will try it out.
|
|
|
|
Joined: Feb 2003
Posts: 306
Fjord artisan
|
OP
Fjord artisan
Joined: Feb 2003
Posts: 306 |
Hi again,
Donno if you have tested it windows 7, but socklisten take considerable longer to start.
Otherwise works like in my Xp server.
Regards
|
|
|
|
|