mIRC Homepage
Posted By: Rewtor Problem with sockets 7.xx - 16/02/11 03:20 PM
I've noticed an issue with some of my scripts that use sockets since using 7.xx where the data doesn't get sent instantly as it is now queued but any other socket events such as sockclose seem to be put ahead of this queue and do not produce an error that the socket didn't receive the sent data.

See example below:

Code:
on *:SOCKLISTEN:TestSocket:{
 sockaccept test.s
 echo -s socket accepted
 sockwrite -tn test.s this is a test
}
on *:SOCKREAD:test*:{
 if ($sockerr > 0) { echo -s error on socket | sockclose $sockname }
 sockread -fn &Read.test | tokenize 32 $bvar(&Read.test,1,$bvar(&Read.test,0)).text
 echo -s $sockname received: $1-
 if ($1 == dotest) {
  sockwrite -tn $sockname testing 1
  sockwrite -tn $sockname testing 2
  sockwrite -tn $sockname testing 3
  sockclose $sockname
 }
}
on *:SOCKWRITE:test*:{
 echo -s $sockname sent data
}
on *:SOCKCLOSE:test*:{
 echo -s $sockname socket closed
}
on *:SOCKOPEN:test*:{
 echo -s $sockname socket opened
 if ($sockname == test) { sockwrite -tn test dotest }
}
alias testsocket {
 socklisten TestSocket 1010
 sockopen test localhost 1010
}
alias sockwrite {
 echo -s data sent to: $2 was: $3-
 sockwrite $1-
}


Result from 7.xx:

Code:
socket accepted
data sent to: test.s was: this is a test
test socket opened
data sent to: test was: dotest
test.s sent data
test sent data
test received: this is a test
test.s received: dotest
data sent to: test.s was: testing 1
data sent to: test.s was: testing 2
data sent to: test.s was: testing 3
test socket closed


Result from earlier versions (6.35):

Code:
test socket opened
data sent to: test was: dotest
socket accepted
data sent to: test.s was: this is a test
test sent data
test.s sent data
test received: this is a test
test.s received: dotest
data sent to: test.s was: testing 1
data sent to: test.s was: testing 2
data sent to: test.s was: testing 3
test received: testing 1
test received: testing 2
test received: testing 3
test socket closed


Is this intentional? If not it means I will need to re-write my scripts that use sockets
Posted By: 5618 Re: Problem with sockets 7.xx - 16/02/11 03:34 PM
Your version of 7.x matters greatly in this case? What version did you use?
Also see this previous bug report.
Posted By: Rewtor Re: Problem with sockets 7.xx - 16/02/11 03:41 PM
I have seen that previous bug report and it still isn't fixed. I have used all 7.xx versions including 7.17 and my script posted still has the issue.

Did you even test it out?
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 11:36 AM
Does anyone know if this is intentional behaviour? To me it is a bug, why isn't the data being sent, not even the first sockwrite even though it is called before the closing of the socket?
Posted By: hixxy Re: Problem with sockets 7.xx - 17/02/11 11:40 AM
Seems intended to me.

You perform /sockwrite and the data is added to a queue.
You then immediately /sockclose before mIRC has chance to process the queue. The help file doesn't suggest that /sockclose is also queued, as there is no need for it to be.

mIRC is a single-threaded application and will process the rest of the sockread event before it deals with the send queue.

You should close the socket in the on SOCKWRITE event to do what you're trying to do.
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 11:46 AM
But it has ALWAYS sent it before a sockclose previously, even though it is queued there isn't anything else in the queue and therefore should be sent immediately... IMO everything should be done the order of which it is called.

It is like that in any programming language mSL included apart from these 7.xx updates...
Posted By: Riamus2 Re: Problem with sockets 7.xx - 17/02/11 12:37 PM
I'm not saying it's right or wrong the way it's working, but you can compare it to a timer. If you set something on a timer (i.e. queued) for 2 seconds from now and immediately turn off the timer, what you told it to do won't get done.

/sockclose really should not be sent immediately. It should be sent only when all of the data is retrieved or sent. In your example, as mentioned, that would be in the sockwrite event.

Like I said, I'm not saying it's handling this the best way possible. Many people will assume it will be handled the same way you're expecting. But with it being queued, this doesn't happen. Maybe it can be improved, but a small change in your script solves the problem as well.
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 12:41 PM
I see what your saying, but in theory if you call it before the sockclose it SHOULD be done before the socket is closed, especially since nothing else is using the queue.

Lets say I wanted to fopen a text file write to it and then fclose it, its basically the same thing... the text WILL be written even though fclose was immediately called because it was called AFTER the fwrite
Posted By: Wims Re: Problem with sockets 7.xx - 17/02/11 01:35 PM
Quote:
but in theory if you call it before the sockclose it SHOULD be done before the socket is closed, especially since nothing else is using the queue.
-
Does anyone know if this is intentional behaviour? To me it is a bug, why isn't the data being sent, not even the first sockwrite even though it is called before the closing of the socket?
/sockwrite use a queue, when the command /sockwrite is done, you must not consider that data is sent
Quote:
Lets say I wanted to fopen a text file write to it and then fclose it, its basically the same thing... the text WILL be written even though fclose was immediately called because it was called AFTER the fwrite
The difference is that /fopen /fwrite and /fclose will block mirc until the file is opened/closed/written, /sockwrite won't since it's queued.

There have been some changes in socket with the 7.x versions, and unicode didn't help speeding the engine. The help file states that mirc try to send the data as fast as possible, it just seems that 6.35 was able to handle your three /sockwrite (or your use of socket in general, since you didn't see that earlier) before /sockclosing the socket.

Posted By: hixxy Re: Problem with sockets 7.xx - 17/02/11 02:13 PM
Originally Posted By: /help /sockwrite
/sockwrite [-tnb] <name> [numbytes] <text|%var|&binvar>

The /sockwrite command queues info to be sent on the specified connection. mIRC will then try to send that info as quickly as it can. Once it has finished sending the info, it triggers the on sockwrite event so you can send more info if you need to.


This has been the description for the sockwrite command as long as I can remember. If the data was sent immediately there would be no point in the on sockwrite event, would there? You could just assume that as soon as the /sockwrite command finished the data was sent.

The behaviour may be different to what it was in previous versions, I don't know, but closing a socket immediately after attempting to send data is bad practice anyway. Use the on sockwrite event, that's what it's there for.
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 02:22 PM
But that would mean my code would need to be longer because the sockwrite event doesn't pass what was written and to where, you are required to do this by redefining the sockwrite alias and will make things messy.

I understand that there is a queue involved but as I said before if nothing is in the queue why isn't it sent instantly?

And my use of fopen etc was just an example to show that it should be processed in the order it is set otherwise it is unreliable, you could have someone on a custom banlist and have your message to the channel stating they are banned with a kick afterwards but the kick would be done first... It's just not logical.

But if this is how Khaled wants the sockwrite to work at least add a switch so we can bypass the queue?
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 02:27 PM
Originally Posted By: hixxy
Originally Posted By: /help /sockwrite
/sockwrite [-tnb] <name> [numbytes] <text|%var|&binvar>

The /sockwrite command queues info to be sent on the specified connection. mIRC will then try to send that info as quickly as it can. Once it has finished sending the info, it triggers the on sockwrite event so you can send more info if you need to.


This has been the description for the sockwrite command as long as I can remember. If the data was sent immediately there would be no point in the on sockwrite event, would there? You could just assume that as soon as the /sockwrite command finished the data was sent.

The behaviour may be different to what it was in previous versions, I don't know, but closing a socket immediately after attempting to send data is bad practice anyway. Use the on sockwrite event, that's what it's there for.


I know that has been in the help file for ages (for as long as I can remember also), but the queues WERE still working in 6.35 but the info was sent instantly IF there wasn't a queue but if loads of data had been sent the $sock().sq would build up so it WAS correct behaviour before 7.xx

Also there isn't any point to the sockwrite event because the only information you can get from it is anything to do with $sock() and $sockname, not what was actually written.

So trying to perform several sockwrites on multiple sockets you wouldn't be able to deal with what was needed after the sockwrites, this is bad for people that use socket connections etc..
Posted By: DJ_Sol Re: Problem with sockets 7.xx - 17/02/11 02:32 PM
As said before and in my opinion, instead of Khaled adding a switch to sockwrite, I think you should not close the socket until you are finished with it.

Another thing I noticed, I dunno, maybe you were testing, but

$bvar(&Read.test,1,$bvar(&Read.test,0)).text

should be the same as: $bvar(&Read.test,1-).text
Posted By: hixxy Re: Problem with sockets 7.xx - 17/02/11 02:34 PM
If you need to do something with the data that was sent then just make a new alias. This is what I used to do:

Code:
alias send {
  if ($sock(blabla)) { 
    sockwrite -n blabla $1- 
    if ($window(@debug)) { aline @debug >>> $1- }
  }
}


Then you just call /send <data>

This behaviour is very similar as to why /sockwrite won't work directly after /sockopen and why you have to wait for the on SOCKOPEN event. mIRC is a single-threaded application and doesn't begin to process the socket messages until it has finished processing the current script.

I'm not even sure how/why it used to work as you describe.
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 02:46 PM
Originally Posted By: DJ_Sol
As said before and in my opinion, instead of Khaled adding a switch to sockwrite, I think you should not close the socket until you are finished with it.

Another thing I noticed, I dunno, maybe you were testing, but

$bvar(&Read.test,1,$bvar(&Read.test,0)).text

should be the same as: $bvar(&Read.test,1-).text


Thanks for the bit below, didn't realise you could do that with binvars. As for not closing until I am finished with it, I AM finished after I have sent the data I need to which is why it is then closed.

@hixxy that is basically the same as rewriting the sockwrite alias as done in the script above to see what had been sent. That is the ONLY way you can see what data was sent, even then the echo would appear once sockwrite was called NOT when it was sent so actually both the rewriting of this alias and the sockwrite event are practically useless in checking what has been SENT in order to continue what you need to do...

For as long as I have used mIRC this has always worked fine, up until the newer 7.xx versions and I don't see the point in rewriting my scripts just to make them even longer than what they are now and even then they won't be very accurate because I can't check when it is sent.

It would easily make a 500 line file 1000 lines.. I know I won't be using any 7.x versions if this is how it's going to stay.
Posted By: Khaled Re: Problem with sockets 7.xx - 17/02/11 03:36 PM
This is the intended and expected behaviour and applies to older versions of mIRC as well.

This issue may be more apparent in newer versions of mIRC due to a few changes to how sockets are handled, however it is in fact present in all versions of mIRC.

Sockets are asynchronous, which means that they send data in the background when they can. There is no guarantee that data can be sent immediately in Windows sockets. An attempt to send data may in fact block, or send only partial data, in which case mIRC will try to send any remaining data when Windows indicates it is ready for more.

When using sockets, you should never assume that you can close a socket immediately after sending data. You should wait until on SOCKWRITE has triggered, as described in the help file, and check the send queue to see if it is empty before closing the socket.
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 03:41 PM
Originally Posted By: Khaled
This is the intended and expected behaviour and applies to older versions of mIRC as well.

This issue may be more apparent in newer versions of mIRC due to a few changes to how sockets are handled, however it is in fact present in all versions of mIRC.

Sockets are asynchronous, which means that they send data in the background when they can. There is no guarantee that data can be sent immediately in Windows sockets. An attempt to send data may in fact block, or send only partial data, in which case mIRC will try to send any remaining data when Windows indicates it is ready for more.

When using sockets, you should never assume that you can close a socket immediately after sending data. You should wait until on SOCKWRITE has triggered, as described in the help file, and check the send queue to see if it is empty before closing the socket.


Hi Khaled, thanks for looking into this however, I already mentioned that queueing was working in older versions but it was still sent instantly if nothing was in the queue, why doesn't 7.xx do this? Surely if nothing is in the queue then it should send when it is called?

And using the sockwrite event I cannot see what was actually wrote to the socket, therefore I cannot perform specific actions based on what was sent plus this would double the size of a script because I will be checking what needs to be sent and then checking it again once it has been sent and performing the necessary

edit: also why are all sockets treated as asynchronous? UDP sockets are asynchronous where as TCP should be synchronous because both sides need to make acknowledgements?
Posted By: Wims Re: Problem with sockets 7.xx - 17/02/11 05:19 PM
Quote:
I already mentioned that queueing was working in older versions but it was still sent instantly if nothing was in the queue, why doesn't 7.xx do this?
I more or less explained that, 6.35 was just faster than 7.xx, you were just lucky that your socket code worked
Quote:
And using the sockwrite event I cannot see what was actually wrote to the socket, therefore I cannot perform specific actions based on what was sent plus this would double the size of a script because I will be checking what needs to be sent and then checking it again once it has been sent and performing the necessary
As stated several times, the on sockwrite event only tell you when you can send informations again, I think Khaled couldn't fill $1- with what was sent even if he wanted to, not sure though. We are just telling you that it's not a bug because it's how sockets work, size of scripts doesn't matter.
Quote:
also why are all sockets treated as asynchronous? UDP sockets are asynchronous where as TCP should be synchronous because both sides need to make acknowledgements?
asynchronous doesn't mean "not connected", it means that a call to a socket function won't lock the program.
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 07:36 PM
Well either way this isn't the way I have always used sockets with mIRC along with MANY other people, just look for a popular script that uses sockets. SO i'll just stick to 6.35
Posted By: hixxy Re: Problem with sockets 7.xx - 17/02/11 07:43 PM
Originally Posted By: Rewtor
Well either way this isn't the way I have always used sockets with mIRC along with MANY other people, just look for a popular script that uses sockets. SO i'll just stick to 6.35


The thing is, as Khaled said, "This is the intended and expected behaviour and applies to older versions of mIRC as well."

In other words the older versions behaved the same way. It's sheer luck that your script runs fast enough to send the data before calling /sockclose, but it won't be like that for everyone who uses your script.

Why are you being so stubborn about this, what's so hard about using the on sockwrite event?
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 07:49 PM
Because the queueing seemed to work better for me in 6.35 and below, I have tested my scripts on 4 different PCs and 1 laptop and had the same results with 6.35 so it isn't down to luck.

Try it yourself, and because I can't see what data was sent with the on sockwrite event so if I wrote multiple things to a socket and had a different action for each one I cannot check....
Posted By: hixxy Re: Problem with sockets 7.xx - 17/02/11 08:33 PM
Khaled himself said that previous versions behave the same way. He has access to the source code. He is the only one who can say with any certainty whether they behave the same way.

Can you name an example when you actually need to know what you've sent via a socket? It's usually the response you should be interested in. If you tell us what you're trying to do we might be able to suggest a better way of doing it.
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 09:04 PM
There is obviously something different in the 7.xx versions for it to only happen in these.

And just one example is sending some information to a server and then closing the socket so it can be used again but I may sometimes get a certain response from the server so I may want to write some more data and the value sent to the server will be dynamic so it will change at some point.



But I will still need to know what was sent in order to process the next action... and one would be closing the socket
Posted By: argv0 Re: Problem with sockets 7.xx - 17/02/11 09:07 PM
Originally Posted By: Rewtor
Because the queueing seemed to work better for me


This is generally not a reason to avoid best practices. As stated by Khaled and other users here, you CANNOT RELY on /sockwrite to immediately write data. You never could. The fact that it worked in 6.35 or prior versions means you were getting lucky, but this is not part of the spec. Implementation details that changed in subsequent versions made your luck disappear.

To answer your initial question: Yes, you need to update your script to work properly. If you follow the SPEC then your script should work for many versions to come-- just like all the other socket scripts that have been working for me for years.
Posted By: argv0 Re: Problem with sockets 7.xx - 17/02/11 09:11 PM
Then only perform a /sockclose in your ON SOCKWRITE event (after setting a flag or using /sockmark to let you know you can close the socket). For instance:

Code:
on *:SOCKOPEN:mysock: {
  var %i = 1
  while (%i < 500) {
    sockwrite -tn $sockname WRITING IMPORTANT DATA!!!
    inc %i 
  }
  sockmark $sockname done
}

on *:SOCKWRITE:mysock: {
  if ($sock($sockname).sq == 0 && $sock($sockname).mark == done) {
    sockclose $sockname
  }
}


This is how it should have been done from the start.
Posted By: Khaled Re: Problem with sockets 7.xx - 17/02/11 09:18 PM
All versions of mIRC queue data with /sockwrite. It may be that older versions of mIRC attempted to send the data immediately however there would still have been no guarantee that it was sent. That is why it is important that you wait for an on SOCKWRITE to check whether your data was actually sent.

So, for example, you can issue multiple /sockwrite requests and when the on SOCKWRITE triggers you can check the $sock().sq to see if there is anything left in the send queue. If the send queue is zero, all the data has been sent and you can now close the socket.

If you do not do this in older versions of mIRC, you will find that your script may work most of the time but it will not be reliable since it will occasionally not send all of the data as expected.
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 09:18 PM
I'm currently checking $sockbr to see if everything has been read and checking $sock().sq to see if it has REACHED it's maximum value if so queue it in a timer to send again... BUT the above still does not let me see what data was sent in the sockwrite event which is what I would need to be able to do if I need to change my scripts for the new versions
Posted By: drum Re: Problem with sockets 7.xx - 17/02/11 09:43 PM
It sounds to me like the real issue here is you aren't sure how to script it the proper way. Basically all you need to do is use a /sockmark instead of /sockclose after the final /sockwrite. This sockmark is the way you tell you are done writing to the socket and it can be closed when writing has finished. See the example that argv0 gave earlier because that's exactly what it does, and it doesn't add that many lines to your code.
Posted By: Rewtor Re: Problem with sockets 7.xx - 17/02/11 10:02 PM
Oh ok I see what you're saying now something along the lines of...

Code:
on *:SOCKREAD:*:{
 ;read the data and whatever...
 if ($1 == whatever) { sockwrite -tn $sockname blahdeblah | sockmark $sockname close }
 elseif ($1 == blahdeblah) { sockwrite -tn $sockname whatever | sockmark $sockname dontclose }
}
on *:SOCKWRITE:*:{
 if ($sock($sockname).mark == close) { sockclose $sockname }
 elseif ($sock($sockname).mark == dontclose) { $nextaction }
}


That makes sense now. Sorry, was just a bit frustrated that it worked before but it doesn't anymore.
Posted By: drum Re: Problem with sockets 7.xx - 17/02/11 10:32 PM
Yes, something along those lines. However you should still check that $sock($sockname).sq == 0 before closing because ON SOCKWRITE can trigger in different conditions:

Originally Posted By: /help on sockwrite
The sockwrite event is triggered when mIRC has finished sending all of the data that you previously queued for sending or when the socket is ready for more writing.
Posted By: tontito Re: Problem with sockets 7.xx - 18/02/11 07:57 PM
Hello,

I believe i was the one that initially reported this issue.
Some optimizations where done after my feedback and i can confirm that all together, mirc 7.17 is faster then mirc 6.35.

I am not sure when sending a few byte each time, but when sending files, i know what i am talking about.

You just have to make sure you use the perfect packet size (depends where you want to send it) when sending to buffer and keep the buffer always with something (meaning you aren't wasting the chance to send something in between).

I have done some research on this and i am able to go up to 4MB/s in data transfer, it is not much but it is a lot for mirc.

Regards
Posted By: argv0 Re: Problem with sockets 7.xx - 18/02/11 09:15 PM
Originally Posted By: tonito
I have done some research on this and i am able to go up to 4MB/s in data transfer, it is not much but it is a lot for mirc.


There have been many claims that mIRC is slow when it comes to transfers. Given your research is showing some pretty decent (amazing if you mean MB == MBYTE and not mbit) results, it would be great if you could post a complete article / guide on the methods you're using to get this kind of speed. I think it would be helpful to other scripters who are having these speed complaints.
Posted By: RusselB Re: Problem with sockets 7.xx - 18/02/11 09:20 PM
For file transfers, mIRC's abilities have never been, and, imo, probably never will be the fastest method.

FTP is designed for file transfers and is, in my experience, a lot faster.

4MB/s (megabyte, not megabit) transfers are, in my experience, an excellent transfer rate for mIRC. Congratulations on obtaining this rate.
Posted By: tontito Re: Problem with sockets 7.xx - 18/02/11 10:46 PM
Hi again,

Most of my research was obtained working on this script Web Server as a result most of what i have learned since i started playing with sockets was placed in it as well.

What i have told you about trying to keep the socket buffer as full as possible and send the right packet size is the result of that research.
Another tip is to keep the cpu usage as low as possible.

I haven't been working much on it lately but my latest test during the 7.16 and 7.16 benchmark shown improvements on speed.

My private version of this script uses something like congestion avoidance algorithm to find in real-time the best size to optimize speed transfer.

You can also see for your self using that addon and converting line 1039 (if i am not mistaken) to a global var, then change the values until you get maximum performance.
I can tell you it is by default optimized for internet transfers.

You can try to find the optimal lan value, the one that allows transfers up to 4 MB/s (and yes, it is megabytes/second).

Regards
Posted By: FroggieDaFrog Re: Problem with sockets 7.xx - 19/02/11 05:23 AM
As reguards to the whole buffer not sending imiately and you having to add a bunch of code and recode built in functions of mIRC to get the socket to do what you wanted, I did it by adding a single line to your code:

Code:
on *:SOCKLISTEN:TestSocket:{
 sockaccept test.s
 echo -s socket accepted
 sockwrite -tn test.s this is a test
}
on *:SOCKREAD:test*:{
 if ($sockerr > 0) { echo -s error on socket | sockclose $sockname }
 sockread -fn &Read.test | tokenize 32 $bvar(&Read.test,1,$bvar(&Read.test,0)).text
 echo -s $sockname received: $1-
 if ($1 == dotest) {
  sockwrite -tn $sockname testing 1
  sockwrite -tn $sockname testing 2
  sockwrite -tn $sockname testing 3
 }
}
on *:SOCKWRITE:test*:{
 echo -s $sockname sent data
 if (!$sock($sockname).sq) { echo -s $sockname Done sending data, closing socket! | sockclose $sockname }
}
on *:SOCKCLOSE:test*:{
 echo -s $sockname socket closed
}
on *:SOCKOPEN:test*:{
 echo -s $sockname socket opened
 if ($sockname == test) { sockwrite -tn test dotest }
}
alias testsocket {
 socklisten TestSocket 1010
 sockopen test localhost 1010
}
alias sockwrite {
 echo -s data sent to: $2 was: $3-
 sockwrite $1-
}


but if you wanted to be even more precise, you could check $sock().rq and $sock().secs before closing it....
Posted By: tontito Re: Problem with sockets 7.xx - 13/05/11 11:42 AM
Hello again Khaled,

Can we open this discussion?
I have been noticing some heavy load on the clients side (browser) and I did some packet analyzing.

It seems that mirc, even if receiving packets of 4 to 8kB via sockwrite, it only sends much smaller packets to the clients, even if the buffer as like 8KB or more of data to send.

I have noticed sizes that vary from 700B to 1456B (never passing this value, I see a lot of reassembles on wireshark).
Is this a max fixed limit or something on the TCP level?

Since most of the application can send 4k I was hoping mirc could as well or eventually up to 8k.

Best regards

Posted By: argv0 Re: Problem with sockets 7.xx - 15/05/11 05:04 AM
tontito:

If you're inspecting data at the packet level and are confused about seeing 1400byte packets, you should familiarize yourself with the MTU.
Posted By: tontito Re: Problem with sockets 7.xx - 15/05/11 05:49 PM
Thanks for reminding me of this.

The high cpu usage in the browser has to have some other explanation.

Regards
© mIRC Discussion Forums