|
Joined: Aug 2003
Posts: 144
Vogon poet
|
OP
Vogon poet
Joined: Aug 2003
Posts: 144 |
Hi ppl. I am traying to return all output data from a game site. In this case it´s from the statistics page. All output data is between <td and </td>, sow i am using the $bfind to search all that data. All the incoming data is about 32 Kb. The very strange thing is that the sockread some times is scrambling the data received, making repeat some output values. I will try to explain with real examples... The images will be showed only the bug error itself and one line above and other line under (between the error). In the Image1 you have: You have to look at the "status window", at the "@OutputData window" and at the small firefox window will show the html source code. The other is not very necessary but you can see it for more information line 15 It´s ok line 16 Not ok, has you can see the last 4 chars are "ana:" and on the source code you will noted that after the "</tr>" is "</tbody>". I think for any reason he have jump out to the final output value off line 15 the word "Registados na última semana:" and put thoses 3 last char´s on this line. line 17 Not ok, has we have noted on the last line he have jump out to one line before and sow he have continued on that line, sow he will repeat all. line 18 Not ok, again it´s the same. It continues. line 19 All it´s ok. This is the correct line for output line 20 All it´s ok line 21 All it´s ok. The "class="tableheader"" have exchange placed with the "align="center"" becouse the firefox source code view have exchange it. Has you can see on this firts example the mirc is scramble the data received After you make again the connection to retrive all data, the error can be in other places and that can depend´s off the machines you have.I have already ask to "Sais" on DaLnet to run the script, when he have done it the errors aren´t in the same place has mine, sow i figured out that mybe the mirc program make thoses randon errors. I will put more 2 or 3 examples images only. The code part that make those output´s are :
on *:SockRead:Game*: {
set -u0 %Bytes 800
sockread $var( %Bytes, 1).value &data
if ( $SockName == GameStats ) {
bwrite $+( $SockName, .html) -1 -1 &data
if ( $gettok( $bvar( &data, 1, %Bytes).text, 1, 13) == HTTP/1.1 302 Found ) {
$iif( %GameMsgInfo == on, msg %GameSetChan, echo -s) You need to make the login $+ $chr(44) Automatic redirect to do it.
Sockclose Game*
SockOpen GameGetCookie xxxxxxx.xxx.xx 80
}
Else {
window @Status 1 1 1135 200
window @OutputData 1 201 1135 200
;Here it will start the search code
;This if is the add the incomplet code missing that can´t be correct read btween the Sockread
if ( $hget( DataIncomplecta, Estatisticas) != $null ) {
bset &DataTemp 1 $hget( DataIncomplecta, Estatisticas)
bset -t &Data 1 $+( $bvar( &DataTemp, 1-).text, $bvar( &Data, 1-).text)
echo -s Data Added 4 $bvar( &DataTemp, 1-).text
hfree DataIncomplecta
}
set %x 1
;while the beguining text "<td" is found
while ( $bfind( &Data, %x, <td) != 0 ) {
;Sets the x var for the bfind value. Line before this one
set %x $bfind( &Data, %x, <td)
;If the final text "</td>" is found, them the output sentense is complet
if ( $bfind( &Data, %x, </td>) != 0 ) {
;This y var i use to set the lengh off the amout off charsets that i must use on the $bvar identifier.
set %y $calc( $calc( $bfind( &Data, %x, </td>) + 5) - %x)
;This is whery the output text is showed including all the html code, beguining in the "<td" search until the final "</td>"
search
aline -p @Status 12 %TimeCount 3Complet $remove( $bvar( &data, %x, %y).text, $chr(9), $chr(10), $chr(13)) 4<td in 12
$bfind( &Data, %x, <td) and 4</td> in12 $bfind( &Data, %x, </td>)
aline -p @OutputData 12 %TimeCount $remove( $bvar( &data, %x, $calc( %y + 60)).text, $chr(9), $chr(10), $chr(13))
TextoSaida $remove( $bvar( &data, %x, %y).text, $chr(9), $chr(10), $chr(13))
inc %TimeCount
}
;;If the final text "</td>" is NOT found, them the output sentense is NOT complet
if ( $bfind( &Data, %x, </td>) == 0 ) {
;Once the final text was not found, it will be ncessary keep the rest off the sentence somewhery, in this case in hash
tables
;It will keep all text from the beguinning "<td" search until the end off sockread data
aline -p @Status 12 %TimeCount 4Incomplet $bvar( &Data, %x, $bvar( &Data, 0)).text 4<td in 12 $bfind( &Data, %x, <td)
and 4</td> in12 $bfind( &Data, %x, </td>) 4Next line will add this info
hadd -m DataIncomplecta Estatisticas $bvar( &Data, %x, $bvar( &Data, 0))
}
inc %x
}
}
}
}
Other´s two images examples Thanks for all
Last edited by Miguel_A; 18/02/08 02:06 AM.
|
|
|
|
Joined: Oct 2005
Posts: 1,741
Hoopy frood
|
Hoopy frood
Joined: Oct 2005
Posts: 1,741 |
I really have no idea what you are asking.. but from what I can piece together, it looks like you need to change your SOCKOPEN event to change:
HTTP/1.1
to
HTTP/1.0
-genius_at_work
|
|
|
|
Joined: Aug 2003
Posts: 144
Vogon poet
|
OP
Vogon poet
Joined: Aug 2003
Posts: 144 |
hummm.
I think you have not understund... All that is fine... All the command conection, i can make the login and i can make the code that is receiving the information.
But the thing is that the mirc for some reason scramble some times that information. The code that i have put here is not all the code, it´s only the part whery that bug/erros or someting appends. Whery mirc ( i think ) scramble the information.
Genius i am Extreminador on your server if you can get ther i will apreciated... I think it´s more easy to explain the problem. Thanks
|
|
|
|
Joined: Oct 2005
Posts: 1,741
Hoopy frood
|
Hoopy frood
Joined: Oct 2005
Posts: 1,741 |
Change lines like this:
%Sock GET / HTTP/1.1 %Sock POST /game-login.php HTTP/1.1 %Sock GET /compress.php?r=statistics HTTP/1.1
and every other HTTP/1.1
like this:
%Sock GET / HTTP/1.0 %Sock POST /game-login.php HTTP/1.0 %Sock GET /compress.php?r=statistics HTTP/1.0
When you use HTTP/1.1, the webserver (not mIRC) returns the data in clustered mode. HTTP/1.1 sends the data in groups of a specific length, instead of each line by itself. HTTP/1.0 sends data 1 line at a time, separated by $crlf.
Make those changes and see if it works.
-genius_at_work
|
|
|
|
Joined: Dec 2002
Posts: 503
Fjord artisan
|
Fjord artisan
Joined: Dec 2002
Posts: 503 |
Scamble? as in the variable doesn't have all the information for some reason?
You do know that the ON SOCKREAD will only read what's been queued in the receive queue, not necessarily the whole amount of data?
This means that if the page you are retrieving is larger than the packet you received, you'll have to wait until the next (and successive) packets are received before you parse any of the retrieved data.
|
|
|
|
Joined: Aug 2003
Posts: 144
Vogon poet
|
OP
Vogon poet
Joined: Aug 2003
Posts: 144 |
Hi again
When you use HTTP/1.1, the webserver (not mIRC) returns the data in clustered mode. HTTP/1.1 sends the data in groups of a specific length, instead of each line by itself. HTTP/1.0 sends data 1 line at a time, separated by $crlf.
Well i didin´t know that, but that is no reason to append what is appening here, in this case. Has you can see on the event and images, the &data is not reading the information directly , i am using a while to search the beguining off the <td , if the <td exists we will search for the output text searching for </td>. If </td> does not exist´s ( this will mean that the &data lengh is almost at the end ), he will recorde all data from <td search until the end off &data ( on the next sockread he will add that data to &data ( i am using hash tables to record the data ) ). If after the while he found </td> he will parce the information that is between </td ( while search ) and </td> (if search ) to the echo -s. Has you can see on the images examples, the errors can happend at the begguining, middle or end off the &data received. Sow what are you saying can be no reason at all to happen what is happening...
If that was the case, the errors will append between the 2 sockread... and is not this case... When you have said that, i have add a red line to be added on all windows and the 3 firts scramble erros that i have found was always on the same sockread &data and not between two sockread.
Sorry for my bad english , but it´s not my primary language
|
|
|
|
Joined: Dec 2002
Posts: 2,962
Hoopy frood
|
Hoopy frood
Joined: Dec 2002
Posts: 2,962 |
... aline -p @OutputData 12 %TimeCount $remove( $bvar( &data, %x, $calc( %y + 60)).text, $chr(9), $chr(10), $chr(13)) ... That would appear to be the problem if I'm undersanding you correctly.
Spelling mistakes, grammatical errors, and stupid comments are intentional.
|
|
|
|
Joined: Aug 2003
Posts: 144
Vogon poet
|
OP
Vogon poet
Joined: Aug 2003
Posts: 144 |
Hi.
Well nop that line is only to check whery the error is... That aline will extend the data lengh of $bvar... That window was only created to check why the error happend´s, with out that "data extension" i will never figured out this error/bug.
I have made that huge extension to check if the correct html source code was iqual to that line... On that way i have noted that it´s not... becouse somehow he jump back in the middle off the while and/or he add´s a randon automadic data...
The real code is at the firefox window... The code that is rong is on @OutputData window
If you check one and ther you will find diferences like i have said some postes ago.
I am almost sure that this is a mirc program bug, but i was expecting that someone could agree with me...
|
|
|
|
Joined: Mar 2006
Posts: 395
Pan-dimensional mouse
|
Pan-dimensional mouse
Joined: Mar 2006
Posts: 395 |
its not sending it through zipped is it?
I knoe it aint zip, but its called something else over HTTP.
[02:16] * Titanic has quit IRC (Excess Flood)
|
|
|
|
Joined: Aug 2003
Posts: 144
Vogon poet
|
OP
Vogon poet
Joined: Aug 2003
Posts: 144 |
Hi guys.
genius_at_work i have already change the code to get the data unsing the -fn options in SockRead, but sometimes it says line too long.
For what i do know it´s not possible to break that line when it says too long when using the -n option on sockread.
If it is not possible, i have to back to the old code, the one that i have put in the firts post. Sow i will need to someone tell me if it something rong with the code, bocouse i can´t find nothing rong. If nothing it´s rong it can be a bug on mirc program.
But i will wait to see what other experts on the matter can say about it.
I do not want to have a rong conclusion about it.
Last edited by Miguel_A; 19/02/08 05:30 PM.
|
|
|
|
Joined: Oct 2003
Posts: 313
Fjord artisan
|
Fjord artisan
Joined: Oct 2003
Posts: 313 |
Starbucks_mafia re-wrote the code in the binvar read loop, and I tested it with the code you gave me prior to posting this.
As far as I can tell it seems to rectify the problem. I ran his mods one run and your original code the next run. The one place I knew there was a duplication of the data in your version did not exhibit a duplication in his. However, I think you need to verify this for yourself. When he's happy with it, he'll post his ammendment, and you can try it.
I did say I'd do a diff between the two bits of code and see if I could understand the reason his changes corrected the issue (we aren't certain why at present). I'll do that when I get a free moment.
Sais
|
|
|
|
Joined: Dec 2002
Posts: 2,962
Hoopy frood
|
Hoopy frood
Joined: Dec 2002
Posts: 2,962 |
on *:SockRead:Game*: {
set -u0 %Bytes 800
sockread %Bytes &newdata
if ($SockName == GameStats) {
bwrite $+( $SockName, .html) -1 -1 &newdata
if ( $gettok( $bvar(&newdata, 1, %Bytes).text, 1, 13) == HTTP/1.1 302 Found ) {
$iif( %GameMsgInfo == on, msg %GameSetChan, echo -s) You need to make the login $+ $chr(44) Automatic redirect to do it.
Sockclose Game*
SockOpen GameGetCookie xxxxxxx.xxx.xx 80
}
else {
window @Status 1 1 1135 200
window @OutputData 1 201 1135 200
; This if is the add the incomplete code missing that can´t be correct read btween the Sockread
if ($hget(DataIncomplecta, Estatisticas)) {
bset &data 1 $hget(DataIncomplecta, Estatisticas)
bcopy &data $bvar(&data, 0) &newdata 1 -1
hfree DataIncomplecta
}
else {
bcopy &data 1 &newdata 1 -1
}
var %open = 1
; While the beginning text "<td" is found
while ($bfind(&data, %open, <td)) {
%open = $v1
; If the final text "</td>" is found, then the output sentence is complete
if ($bfind(&data, %open, </td>)) {
var %close = $v1
var %len = $calc( %close + 5 - %open)
;This is where the output text is showed including all the html code, beguining in the "<td" search until the final "</td>"
var %celldata = $remove( $bvar( &data, %open, %len).text, $chr(9), $chr(10), $chr(13))
aline -p @Status 12 %TimeCount 3Complet %celldata 4<td in 12 %open and 4</td> in12 %close
aline -p @OutputData 12 %TimeCount %celldata
TextoSaida %celldata
inc %TimeCount
}
; If the final text "</td>" is NOT found, them the output sentence is NOT complete
else {
; Once the final text was not found, it will be necessary keep the rest of the sentence somewhere, in this case in hash tables
; It will keep all text from the beginning "<td" search until the end of sockread data
bcopy &incomplete 1 &data %open -1
hmake DataIncomplecta 10
noop $hget(DataIncomplecta, Estatisticas, &incomplete)
aline -p @Status 12 %TimeCount 4Incomplete $bvar( &incomplete, %open -).text 4<td in 12 %open
}
inc %open
}
}
}
} I did a lot of chopping and changing in the code so there's a slight chance I've removed some things and forgotten to put them back in. It should be about right though - at least enough to test whether the bug is still present. I also renamed some of the variables to make it easier for me to work on the code so you might want to rename them ( %x is now %open, %y is now %len). I also added a few variables which cut down on a lot of redundant re-evaluating. Even if you rename the variables you should consider keeping them - they make the code more efficient.
Spelling mistakes, grammatical errors, and stupid comments are intentional.
|
|
|
|
Joined: Aug 2003
Posts: 144
Vogon poet
|
OP
Vogon poet
Joined: Aug 2003
Posts: 144 |
Hi all. A special thanks to starbucks_mafia and Sais.
Firts off all, thanks for spending time on the code, but unfortunately it happened the same thing. I think that at this time we can consider that this is a mirc program bug, all the thing point to a bug in the $bvar and/or $bfind identifier, i don´t think that the bug can be in the sockread but we can consider it too.
But i will make a Bug Tester Script using any other site/page online, to paste and to all can check if i am right or not.
|
|
|
|
|