<p>It is a research problem called <strong>wrapper induction</strong> or <strong>web data extraction</strong>. I don't know any library for this, but there are a lot of research papers (see below the list of good ones IMHO) and some research projects like <a href="http://diadem.cs.ox.ac.uk/" rel="nofollow">DIADEM</a> (their site contains list of publications as well).</p>

<ul>
<li>Muslea, Ion, Steven Minton, and Craig A. Knoblock. “<a href="http://www.ai.sri.com/~muslea/PS/jaamas-2k.pdf" rel="nofollow">Hierarchical
Wrapper Induction for Semistructured Information Sources</a>.” Autonomous
Agents and Multi-Agent Systems 4, no. 1–2 (2001): 93–114. </li>
<li>Dalvi, Nilesh, Ravi Kumar, and Mohamed Soliman. “<a href="http://arxiv.org/pdf/1103.2406" rel="nofollow">Automatic Wrappers for
Large Scale Web Extraction.</a>” Proceedings of the VLDB Endowment 4, no.
4 (2011): 219–230. </li>
<li>Dalvi, Nilesh, Ashwin Machanavajjhala, and Bo Pang. “An Analysis of
Structured Data on the Web.” Proceedings of the VLDB Endowment 5, no.
7 (2012): 680–691.</li>
<li>Gentile, Anna Lisa, Ziqi Zhang, Isabelle Augenstein, and Fabio
Ciravegna. “<a href="http://arxiv.org/pdf/1203.6406" rel="nofollow">Unsupervised Wrapper Induction Using Linked Data</a>.” In
Proceedings of the Seventh International Conference on Knowledge
Capture, 41–48, 2013.   </li>
<li>Weninger, Tim, and Jiawei Han. “Exploring Structure and Content on
the Web: Extraction and Integration of the Semi-Structured Web.” In
Proceedings of the Sixth ACM International Conference on Web Search
and Data Mining, 779–780, 2013.
<a href="http://dl.acm.org/citation.cfm?id=2433499" rel="nofollow">http://dl.acm.org/citation.cfm?id=2433499</a>.</li>
</ul>
How to discover common information blocks from multiple web pages of a same website?
<p>It is a pattern recognition task in web crawler. The traditional crawler gets the data of the whole page. If there is any way to make the crawler a litter intelligence, like just to identify and capture the the information part.</p>