mIRC Home    About    Download    Register    News    Help

Print Thread
Joined: Jul 2006
Posts: 4,145
W
Wims Offline OP
Hoopy frood
OP Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 4,145
1) $urlget removes the Content-length header from the $urlget().reply value

2) as a result of 1) $urlget does not report the value of the Content length header via $urlget().size, this value is 0 and it does not create/populate the output filename (if a binvar is used as the output, the binvar is filled correctly, but content length still missing and size still 0)

//.remove test.mrc | alias urlgetfail if (content-length !isin $!urlget($1).reply) echo -ag no content length size: $!urlget($1).size state $!urlget($1).state content $!exists(test.mrc) $!lof(test.mrc) reply $!urlget($1).reply $(|) else echo -ag state $!urlget($1).state $(|) alias urlgetfail | noop $urlget(https://www.mirc.com/versions.txt,gfik,test.mrc,urlgetfail)


#mircscripting @ irc.swiftirc.net == the best mIRC help channel
Joined: Dec 2002
Posts: 5,411
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 5,411
Thanks for your bug report. The most likely reason for this is that the content-length is not always included in the header reply from the server. Did you check to see if the content-length is actually in the header? eg. by opening the link in a browser and using developer tools to see the header details?

Joined: Jul 2006
Posts: 4,145
W
Wims Offline OP
Hoopy frood
OP Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 4,145
It's not possible for the content length header not to be provided by an http server for this situation, no need to check. What was tried was multiple http servers and content length is always removed.


#mircscripting @ irc.swiftirc.net == the best mIRC help channel
Joined: Jul 2006
Posts: 4,145
W
Wims Offline OP
Hoopy frood
OP Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 4,145
Sorry about that previous post, I mistested, you're right, with chunk encoding, no content length header is required, which is what www.mirc.com is doing.

But wrong example then, the issue is still present when chunk encoding is not used, here is a better example:

//alias urlgetfail if (content-length !isin $!urlget($1).reply) echo -ag no content length: $!v2 $(|) alias urlgetfail | noop $urlget(https://raw.githubusercontent.com/SReject/JSON-For-Mirc/master/src/JSON%20For%20mIRC.mrc,gfik,test.mrc,urlgetfail)

vs

alias urlgetfail2 sockclose urlgetfail | sockopen -e urlgetfail raw.githubusercontent.com 443
on *:sockopen:urlgetfail:sockwrite urlgetfail $+(GET /SReject/JSON-For-Mirc/master/src/JSON%20For%20mIRC.mrc HTTP/1.1,$crlf,Host: raw.githubusercontent.com,$crlf,User-Agent: mIRC,$crlf,$crlf)
on *:sockread:urlgetfail:var %a | sockread %a | if (!$sockbr) return | if (%a == $null) sockclose urlgetfail | else echo -ag headers: %a


Originally Posted by /sockopen
headers: HTTP/1.1 200 OK
headers: Connection: keep-alive
headers: Content-Length: 42128
headers: Cache-Control: max-age=300
headers: Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'; sandbox
headers: Content-Type: text/plain; charset=utf-8
headers: ETag: "a98867cff9c5baf701be4c97ea1b39d354e2704dcaccc274cb3bfa9d4212bcff"
headers: Strict-Transport-Security: max-age=31536000
headers: X-Content-Type-Options: nosniff
headers: X-Frame-Options: deny
headers: X-XSS-Protection: 1; mode=block
headers: X-GitHub-Request-Id: F598:1C3F:9B750:D7823:643563E3
headers: Accept-Ranges: bytes
headers: Date: Tue, 11 Apr 2023 13:43:00 GMT
headers: Via: 1.1 varnish
headers: X-Served-By: cache-mrs10579-MRS
headers: X-Cache: MISS
headers: X-Cache-Hits: 0
headers: X-Timer: S1681220580.807887,VS0,VE265
headers: Vary: Authorization,Accept-Encoding,Origin
headers: Access-Control-Allow-Origin: *
headers: X-Fastly-Request-ID: 45c8a7c842858d548578fec2ad9d088b94b45cd5
headers: Expires: Tue, 11 Apr 2023 13:48:00 GMT
headers: Source-Age: 0
Originally Posted by $urlget
no content length: HTTP/1.1 200 OK
Connection: keep-alive
Cache-Control: max-age=300
Content-Security-Policy: default-src 'none'; style-src 'unsafe-inline'; sandbox
Content-Type: text/plain; charset=utf-8
ETag: W/"e3990ff20cc6506d6eb2c5acd28024d73beda98f037186e4b1cabe933637a433"
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
X-GitHub-Request-Id: FE10:3D5E:5056C3:639E88:64356057
Accept-Ranges: bytes
Date: Tue, 11 Apr 2023 13:44:19 GMT
Via: 1.1 varnish
X-Served-By: cache-mrs10549-MRS
X-Cache: HIT
X-Cache-Hits: 2
X-Timer: S1681220659.094595,VS0,VE0
Vary: Authorization,Origin
Access-Control-Allow-Origin: *
X-Fastly-Request-ID: 9ed29e68eac9b65651f7e947a25f9ce5414fa99d
Expires: Tue, 11 Apr 2023 13:49:19 GMT
Source-Age: 285


#mircscripting @ irc.swiftirc.net == the best mIRC help channel
Joined: Dec 2002
Posts: 5,411
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 5,411
No problem, it does actually look like the WinInet API removes the Content-Length from the reply header when the client enables gzip/deflate decoding. As far as I can tell, there is no way to prevent this, as it is part of the API, so you would need to use .size in this context.

Joined: Jul 2006
Posts: 4,145
W
Wims Offline OP
Hoopy frood
OP Offline
Hoopy frood
W
Joined: Jul 2006
Posts: 4,145
Quote
so you would need to use .size in this context
But this is part of my report Khaled, I cannot use .size because it returns 0 because the API is removing the content length. Another thing is that the output file is not created in this case. It seems rather strange that the API would be so that you cannot get the length crazy


#mircscripting @ irc.swiftirc.net == the best mIRC help channel
Joined: Dec 2002
Posts: 5,411
Hoopy frood
Offline
Hoopy frood
Joined: Dec 2002
Posts: 5,411
Quote
But this is part of my report Khaled, I cannot use .size because it returns 0 because the API is removing the content length. Another thing is that the output file is not created in this case. It seems rather strange that the API would be so that you cannot get the length
The output file is not created? That is odd. When I run your above scripts, the file is downloaded correctly. Can you please test this again? As for .size, sorry, I meant .rcvd, which returns the total bytes received.
Quote
It seems rather strange that the API would be so that you cannot get the length
If you find a way around it, let me know. As far as I can tell, it is an API bug. The API that enables gzip/deflate decoding in WinINet, but disables content-length in the header, is InternetSetOption() with the INTERNET_OPTION_HTTP_DECODING option. If INTERNET_OPTION_HTTP_DECODING is not set, content-length is present as expected.

Remember that content-length is the size of the transfer, so if the transfer encoding is gzip/deflate, it will be the compressed transfer size, not the size of the received file, which is in .rcvd.


Link Copied to Clipboard