title: HTML Treebuilder XPath to Extract Links
body: <p>I am writing a basic script which just extracts all the links from a web page. It is written in Perl and makes use of WWW::Mechanize and HTML::Treebuilder::Xpath modules, both of which I have installed through CPAN.</p>
  
  <p>I know it can be easily done using only WWW::Mechanize, however would like to learn to do it using XPath as well.</p>
  
  <p>So, the script will parse the entire web page, and check the href attribute for every anchor tag, extract the link and print it to the console/write it to a file. Please note that in the script below, I have not used use strict, since I am only writing this to clarify and understand the concept of using XPath to traverse the HTML Tree.</p>
  
  <p>here is the script:</p>
  
  <pre><code>#! /usr/bin/perl
  
  use WWW::Mechanize;
  use HTML::TreeBuilder::XPath;
  use warnings;
  
  $url="https://example.com";
  
  $mech=WWW::Mechanize-&gt;new();
  $mech-&gt;get($url);
  
  $tree=HTML::TreeBuilder::XPath-&gt;new();
  
  $tree-&gt;parse($mech-&gt;content);
  
  $nodes=$tree-&gt;findnodes(q{'//a'}); # line is modified later.
  
  foreach $node($nodes)
  {
      print $node-&gt;attr('href');
  }
  </code></pre>
  
  <p>And it gives an error:</p>
  
  <pre><code>Can't locate object method "attr" via package "XML::XPathEngine::Literal" at pagegetter.pl line 23.
  </code></pre>
  
  <p>I have modified the script as follows:</p>
  
  <pre><code>$nodes=$tree-&gt;findnodes(q{'//a/@href'});
  
  while($node=$nodes-&gt;shift)
  {
    print $node-&gt;attr('href');
  }
  </code></pre>
  
  <p>Error:</p>
  
  <pre><code>Can't locate object method "shift" via package "XML::XPathEngine::Literal"
  </code></pre>
  
  <p>I am not sure, how to print the value of the href attribute.</p>
  
  <p>$nodes should hold the list of all the href attributes? I believe it does not store the value but instead pointers to it?</p>
  
  <p>I tried searching and reading examples, however I am not sure how to go about it.</p>
  
  <p>Thanks.</p>