mIRC Home    About    Download    Register    News    Help

Print Thread
Page 2 of 2 1 2
Joined: Mar 2008
Posts: 24
T
tparry Offline OP
Ameglian cow
OP Offline
Ameglian cow
T
Joined: Mar 2008
Posts: 24
Due to the complexity of the html code (20 chunks of data in 60,000 chars), I decided to use a simple perl script to extract the data. It works like a charm, but i'd still prefer to keep everything in mIRC. Too bad mIRC has so many variable limitations. No idea why!

I guess I should learn how to do all the socket programing in perl, then it wont be such a mess.

Joined: Mar 2008
Posts: 24
T
tparry Offline OP
Ameglian cow
OP Offline
Ameglian cow
T
Joined: Mar 2008
Posts: 24
Theres 20 chunks of this code in 1 line of the html code.
Does this look like something easy to manipulate in mIRC? I need only 7 fields from each chunk, but they arent easy to match:

<div class="results" style="width: 395px;"><table><tr><td class="sImage"><a href="profile.php?id=825603469" onclick="(new Image()).src = &#039;/ajax/ct.php?app_id=7906852977&action_type=3&post_form_id=170baa3572aa7ee38b8b9f6fa0f04676&position=3&&#039; + Math.random();return true"><img size="s" uid="825603469" linked="no" src="http://profile.ak.facebook.com/profile5/473/88/s825603469_7283.jpg" alt="Sarah Bockus" title="Sarah Bockus" /></a></td><td class="sInfo"><font class="bigger" style="line-height: 16px;"><b><a href="profile.php?id=825603469" onclick="(new Image()).src = &#039;/ajax/ct.php?app_id=7906852977&action_type=3&post_form_id=170baa3572aa7ee38b8b9f6fa0f04676&position=3&&#039; + Math.random();return true">Sarah Bockus</a></b></font>
<div class="blurb">&#34;:)"I will fight to the end...lol&#34;</div><table style="line-height: 16px; margin-top: 3px; color: #898989;"><tr><td width="40px">Price:</td><td><b style="color: #006600;">$23,378</b></td></tr><tr><td>Bling:</td><td><b style="color: #006600;">$52,606</b></td></tr><td>Hotties:</td><td><font style="color: red;">12</font></td></table></td><td class="actions" style="width: 145px;"><div class="navLink"><a class="submenu" clicktoshowdialog="my_dialog" onclick="FBML.clickToShowDialog("app7906852977_my_dialog");fbjs_sandbox.instances.a7906852977.bootstrap();return fbjs_dom.eventHandler.call([fbjs_dom.get_instance(this,7906852977),function(a7906852977_event) {a7906852977_getFavor(825603469);},7906852977],new fbjs_event(event));return false"><div class="nav">Add to Favorites</div></a></div><div class="navLink"><a class="submenu" clicktoshowdialog="my_dialog" onclick="FBML.clickToShowDialog("app7906852977_my_dialog");fbjs_sandbox.instances.a7906852977.bootstrap();return fbjs_dom.eventHandler.call([fbjs_dom.get_instance(this,7906852977),function(a7906852977_event) {a7906852977_getPoke(825603469);},7906852977],new fbjs_event(event));return false"><div class="nav">Poke Sarah!</div></a></div><div class="navLink"><a class="submenu" href="gifts.php?id=825603469" onclick="(new Image()).src = &#039;/ajax/ct.php?app_id=7906852977&action_type=3&post_form_id=170baa3572aa7ee38b8b9f6fa0f04676&position=3&&#039; + Math.random();return true"><div class="nav">Give a Present</div></a></div><div class="navLink"><a class="submenu" clicktoshowdialog="my_dialog" onclick="FBML.clickToShowDialog("app7906852977_my_dialog");fbjs_sandbox.instances.a7906852977.bootstrap();return fbjs_dom.eventHandler.call([fbjs_dom.get_instance(this,7906852977),function(a7906852977_event) {a7906852977_getInfo(825603469);},7906852977],new fbjs_event(event));return false"><div class="nav">Buy for <b style="color: #006600;">$26,728</b></div></a></div></td></tr></table></div></b></div></a></div></td></tr></table></div>

Joined: Sep 2005
Posts: 2,881
H
Hoopy frood
Offline
Hoopy frood
H
Joined: Sep 2005
Posts: 2,881
I've copied and pasted that chunk of text into a file and then read it into a binvar to make this code, but it should work no matter how you fill the variable with some small modifications.

Code:
alias parseprofile {
  bread test.txt 0 $file(test.txt) &data
  var %pointer = $bfind(&data,1,profile.php?id=), %profile, %profileimage, %profiletitle, %profileprice, %profilebling, %profilehotties, %profilebuyfor
  while (%pointer) {
    if ($bfind(&data,%pointer,")) { 
      %profile = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
      %pointer = $v1 + 1
    }
    %pointer = $bfind(&data,%pointer,src="http://profile.) + 5
    if ($bfind(&data,%pointer,")) {
      %profileimage = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
      %pointer = $v1
    }
    %pointer = $bfind(&data,%pointer,title=") + 7
    if ($bfind(&data,%pointer,")) {
      %profiletitle = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
      %pointer = $v1
    }
    %pointer = $bfind(&data,%pointer,Price:)
    %pointer = $bfind(&data,%pointer,$)
    if ($bfind(&data,%pointer,<)) {
      %profileprice = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
      %pointer = $v1
    }
    %pointer = $bfind(&data,%pointer,Bling:)
    %pointer = $bfind(&data,%pointer,$)
    if ($bfind(&data,%pointer,<)) {
      %profilebling = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
      %pointer = $v1
    }
    %pointer = $bfind(&data,%pointer,Hotties:)
    %pointer = $bfind(&data,%pointer,red;">) + 6
    if ($bfind(&data,%pointer,<)) {
      %profilehotties = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
      %pointer = $v1
    }
    %pointer = $bfind(&data,%pointer,Buy for)
    %pointer = $bfind(&data,%pointer,$)
    if ($bfind(&data,%pointer,<)) {
      %profilebuyfor = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
      %pointer = $v1
    }
    echo -a * Profile: %profile
    echo -a * Profile image: %profileimage
    echo -a * Profile title: %profiletitle
    echo -a * Profile price: %profileprice
    echo -a * Profile bling: %profilebling
    echo -a * Profile hotties: %profilehotties
    echo -a * Profile buy for: %profilebuyfor
    %pointer = $bfind(&data,%pointer,profile.php?id=)
  }
}


Of course you can do whatever you want with the variables.

Joined: Mar 2008
Posts: 24
T
tparry Offline OP
Ameglian cow
OP Offline
Ameglian cow
T
Joined: Mar 2008
Posts: 24
Oh wow it works, im quite impressed!
The only thing is, I got 19/20 results outputed.
This is because sometimes a profile is private and theres no name included. So this is missing from the code:

alt="name here" title="name here"

Also, the url for those without a title is http://static. and not http://profile.

Still trying to figure out how to fix that.

Last edited by tparry; 24/03/08 02:41 PM.
Joined: Sep 2005
Posts: 2,881
H
Hoopy frood
Offline
Hoopy frood
H
Joined: Sep 2005
Posts: 2,881
If you paste the chunk for a private profile I can take a look.

Joined: Mar 2008
Posts: 24
T
tparry Offline OP
Ameglian cow
OP Offline
Ameglian cow
T
Joined: Mar 2008
Posts: 24
I got it working, thanks! laugh

Code:
alias parseprofile {
  bread hfs.recent.txt 0 $file(hfs.recent.txt) &data
  var %pointer = $bfind(&data,1,profile.php?id=)
  while (%pointer) {
    var %profile, %profileimage, %profiletitle, %profileprice, %profilebling, %profilehotties, %profilebuyfor
    if ($bfind(&data,%pointer,")) { 
      %profile = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
      %pointer = $v1 + 1
      %pointer = $bfind(&data,%pointer,src="http://) + 5
      if ($bfind(&data,%pointer,")) {
        %profileimage = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
        %pointer = $v1
        if (http://profile isin %profileimage) {
          %pointer = $bfind(&data,%pointer,title=") + 7
          if ($bfind(&data,%pointer,")) {
            %profiletitle = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
            %pointer = $v1
          }
        }
        else { %profiletitle = "PRIVATE" }
        %pointer = $bfind(&data,%pointer,Price:)
        %pointer = $bfind(&data,%pointer,$)
        if ($bfind(&data,%pointer,<)) {
          %profileprice = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
          %pointer = $v1
          %pointer = $bfind(&data,%pointer,Bling:)
          %pointer = $bfind(&data,%pointer,$)
          if ($bfind(&data,%pointer,<)) {
            %profilebling = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
            %pointer = $v1
            %pointer = $bfind(&data,%pointer,Hotties:)
            %pointer = $bfind(&data,%pointer,red;">) + 6
            if ($bfind(&data,%pointer,<)) {
              %profilehotties = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
              %pointer = $v1
              %pointer = $bfind(&data,%pointer,Buy for)
              %pointer = $bfind(&data,%pointer,$)
              if ($bfind(&data,%pointer,<)) {
                %profilebuyfor = $bvar(&data,$+(%pointer,-,$calc($v1 - 1))).text
                %pointer = $v1
                echo -a * Profile: %profile
                echo -a * Profile image: %profileimage
                echo -a * Profile title: %profiletitle
                echo -a * Profile price: %profileprice
                echo -a * Profile bling: %profilebling
                echo -a * Profile hotties: %profilehotties
                echo -a * Profile buy for: %profilebuyfor
              }
            }
          }
        }
      }
    }
    %pointer = $bfind(&data,%pointer,profile.php?id=)
  }

Joined: Oct 2003
Posts: 3,918
A
Hoopy frood
Offline
Hoopy frood
A
Joined: Oct 2003
Posts: 3,918
whats wrong with just matching the first href="" for the profile link, first title="" for the name, etc..?



- argv[0] on EFnet #mIRC
- "Life is a pointer to an integer without a cast"
Joined: Sep 2005
Posts: 2,881
H
Hoopy frood
Offline
Hoopy frood
H
Joined: Sep 2005
Posts: 2,881
He said there's 20 of those chunks in one line of html. Matching the first result would ignore the other 19 profiles.

Joined: Oct 2004
Posts: 8,330
Hoopy frood
Offline
Hoopy frood
Joined: Oct 2004
Posts: 8,330
If the lines are too long, then you need binary variables. That said, you *can* manipulate binary variables within mIRC without writing them to a file. Using $bvar allows you to do this. If you check each line for your matchtext, you may not even need to actually work with lines over the ~500 character limit. If that is the case, it won't be very difficult at all to handle the data. If you give a *small* sample of the data that you're actually trying to grab from the site, it would be easy to show you how to match it and use it.


Invision Support
#Invision on irc.irchighway.net
Joined: Mar 2008
Posts: 24
T
tparry Offline OP
Ameglian cow
OP Offline
Ameglian cow
T
Joined: Mar 2008
Posts: 24
Originally Posted By: Riamus2
If the lines are too long, then you need binary variables. That said, you *can* manipulate binary variables within mIRC without writing them to a file. Using $bvar allows you to do this. If you check each line for your matchtext, you may not even need to actually work with lines over the ~500 character limit. If that is the case, it won't be very difficult at all to handle the data. If you give a *small* sample of the data that you're actually trying to grab from the site, it would be easy to show you how to match it and use it.


I posted a sample of 1 chunk a few replies ago. You can copy/paste that 20 times on the same line to replicate the data I receive from the webpage.
Much appreciated smile

Joined: Oct 2004
Posts: 8,330
Hoopy frood
Offline
Hoopy frood
Joined: Oct 2004
Posts: 8,330
Sorry, ignore my reply. I didn't notice the second page and replied to the post at the end of the first page.


Invision Support
#Invision on irc.irchighway.net
Page 2 of 2 1 2

Link Copied to Clipboard