Grabbing info from websites

Print Thread

Grabbing info from websites #99758 05/10/04 01:43 AM
Joined: Oct 2003 Posts: 64 E Exlax OP Babel fish
OP Exlax Babel fish E Joined: Oct 2003 Posts: 64	I need a very BASIC tutorial on getting information from a website. Here is the example I want you to use. A website returns the value 0 or 1. And that is the only thing printed on the page. How would mIRC read this value?

Re: Grabbing info from websites

#99759 05/10/04 01:47 AM

Joined: Nov 2003

Posts: 2,327

tidy_trax

Hoopy frood

tidy_trax

Hoopy frood

Joined: Nov 2003

Posts: 2,327

I'm not very good at tutorials, but here's a commented example, comments are in green, stuff that can be changed is in red.

Code:

/sockopen [color:red]socketname website.com[/color] 80
[color:Green];Open a socket to [color:Red]website.com[/color][/color]
on *:sockopen:[color:red]socketname[/color]:{

  [color:Green];This is called when the socket is opened.[/color]

  if ($sockerr) { return }

  [color:Green];Stop running the code if there is an error when opening the socket.[/color]

  sockwrite -n $sockname GET / HTTP/1.1
  sockwrite -n $sockname Host: [color:red]website.com[/color]
  sockwrite -n $sockname Accept: */*
  sockwrite -n $sockname Connection: Close
  sockwrite -n $sockname

  [color:Green];Send the relative information to get the source of the website.[/color] 
  [color:Green];Change "/" in the "GET / HTTP/1.1" line if you want to get a directory or file instead.[/color]

}
on *:sockread:[color:red]socketname[/color]:{

  [color:Green];This is called when data is read from a socket.[/color]

  var %s
  [color:Green];Make sure "%s" is a local variable.[/color]
  sockread %s
  [color:Red]echo -a[/color] %s

  [color:Green];Read the data into %s, then echo the data[/color]

}

New username: hixxy

Re: Grabbing info from websites #99760 05/10/04 01:55 AM
Joined: Oct 2003 Posts: 64 E Exlax OP Babel fish
OP Exlax Babel fish E Joined: Oct 2003 Posts: 64	That was excellent in how it worked in all. Now, Im a bit interested if somebody could explain the meaning of all the socket-specific commands he used. basically: what it's function would do, and what would happen if it wasnt there

Re: Grabbing info from websites #99761 05/10/04 02:08 AM
Joined: Nov 2003 Posts: 2,327 T tidy_trax Hoopy frood
tidy_trax Hoopy frood T Joined: Nov 2003 Posts: 2,327	/help sockets will tell you that. New username: hixxy

Re: Grabbing info from websites #99762 05/10/04 02:13 AM
Joined: Oct 2003 Posts: 64 E Exlax OP Babel fish
OP Exlax Babel fish E Joined: Oct 2003 Posts: 64	I just finished reading over that, and it barely helped. I think my whole understanding of Sockets is a bit fuzzy. An detailed but consise explanation of what they are and how they work would be great. And what I'm most interested in from your code is: sockwrite -n $sockname GET / HTTP/1.1 sockwrite -n $sockname Host: website.com sockwrite -n $sockname Accept: / sockwrite -n $sockname Connection: Close sockwrite -n $sockname ;Send the relative information to get the source of the website. ;Change "/" in the "GET / HTTP/1.1" line if you want to get a directory or file instead. What exactly is taking place here? And an additional question: What is invoking on *:sockread:socketname: ? EDIT: And I just noticed random numbers are stuck in the source code. Any reason for this? Last edited by Exlax; 05/10/04 02:16 AM.

Re: Grabbing info from websites #99763 05/10/04 04:39 AM
Joined: Mar 2003 Posts: 1,271 Sunnydale, CA L LocutusofBorg Hoopy frood
LocutusofBorg Hoopy frood L Joined: Mar 2003 Posts: 1,271 Sunnydale, CA	That's just the stuff the webserver needs to process your request. If you want to know the specifics, you're going to have to look up the RFC/HTML protocols and read that. DALnet #Helpdesk I hear and I forget. I see and I remember. I do and I understand. -Confucius

Re: Grabbing info from websites #99764 05/10/04 05:57 AM
Joined: Dec 2002 Posts: 788 Wales, United Kingdom. C Coolkill Hoopy frood
Coolkill Hoopy frood C Joined: Dec 2002 Posts: 788 Wales, United Kingdom.	You are best using HTTP/1.0 not /1.1 as suggested above, usually because version 1.1 supports 'chunked data transfer' which you mean you would need to parse them in mIRC, using version 1.0 will prevent them being sent, thus you wont have to parse them.. As for what the jargon does.. GET / HTTP/1.1 / is the page you wish to 'get' or 'recieve' using just a / will return the default page, usually index.php/html/htm, you can also do, /mydir/mypage.html to get a specific page. HTTP/1.1 is the version of protocol you are using (or support, 1.1 being the most recent and 1.0 being the basics, no frills version. Accept / tells the server what type of files you want to recieve, / being anything. connection allows you to tell the server what you want to do when your finished, i.e. close the connection. Eamonn.

Re: Grabbing info from websites #99765 05/10/04 06:03 AM
Joined: Jun 2003 Posts: 5,024 London, England M Mentality Hoopy frood
Mentality Hoopy frood M Joined: Jun 2003 Posts: 5,024 London, England	You may wish to view this tutorial, it's helped quite a lot of people and it goes into a bit more detail. Regards, Mentality/Chris

Re: Grabbing info from websites #99766 05/10/04 06:19 AM
Joined: Nov 2003 Posts: 2,327 T tidy_trax Hoopy frood
tidy_trax Hoopy frood T Joined: Nov 2003 Posts: 2,327	I'm not saying the webserver couldn't send chunked data anyway, but i've never recieved a chunked reply unless the data I sent to it used "transfer-encoding: chunked". New username: hixxy

Link Copied to Clipboard

Forums Scripts & Popups Grabbing info from websites