title: htmlcxx c++ crawling html
body: <p>I've read many different issues from different people regarding libraries to crawl HTML. I've decided to go with htmlcxx, because it looks simple and it's in the Ubuntu repository.. Anyhow, while playing with htmlcxx I was trying to achieve a simple task and grabbing the text between heading tags. Using the iterator, it->text() returns the tag itself and it->textClosing() returns the closing of the tag. My question is, how can I get at the data BETWEEN the tags? I mean, there must be a way, why make a library to crawl html and not have this functionality? If anyone can point me in the right direction, I'd appreciate it.</p>
  
  <p>You can check out what I've done with svn so far with:
  svn co svn://yunices.dyndns.org/repository/nich/trunk .</p>
  
  <p>or view through websvn:
  <a href="https://yunices.dyndns.org/" rel="nofollow">https://yunices.dyndns.org/</a></p>
  
  <p>Here's the particular snippet in question:</p>
  
  <pre><code>void node::get_headings() {
      tree&lt;htmlcxx::HTML::Node&gt;::iterator it = dom.begin();
      tree&lt;htmlcxx::HTML::Node&gt;::iterator end = dom.end();
      for (; it != end; ++it) {
          static const boost::regex expression("[h|H][1-6]");
          if(boost::regex_search(it-&gt;tagName(), expression)) {
              it-&gt;parseAttributes();
              std::cout &lt;&lt; it-&gt;text() &lt;&lt; "&lt;=&gt;" &lt;&lt; it-&gt;closingText() &lt;&lt; std::endl;
              std::map&lt;std::string, std::string&gt; pairs = it-&gt;attributes();
              for ( std::map&lt;std::string, std::string&gt;::const_iterator iter = pairs.begin();
                      iter != pairs.end(); ++iter ) {
                  std::cout &lt;&lt; iter-&gt;first &lt;&lt; ":" &lt;&lt; iter-&gt;second &lt;&lt; "\n";
              }
          }
      }
  }
  </code></pre>