title: Nutch Crawling Using Regex
body: <p>I wanted to crawl a website but save only those web pages ,
  which are like :</p>
  
  <p>Type A : <a href="http://mywebsite.com/page.html" rel="nofollow">http://mywebsite.com/page.html</a></p>
  
  <p>any other links like. : <a href="http://mywebsite.com/a/b/" rel="nofollow">http://mywebsite.com/a/b/</a> or <a href="http://mywebsite.com/a/" rel="nofollow">http://mywebsite.com/a/</a>
  or any thing like this should be in queue to be crwaled , but they should be just used to get more url likes "type A" but not stored .</p>
  
  <p>How to use regular expression in regex-urlfilter.txt in nutch .</p>
  
  <p>or in RegexUrlNormalizer.xml</p>
  
  <p>" I think that crawling on whole would be better option . But I also want that  while crawling . If a URL has anchor links like Type A : <a href="http://mywebsite.com/page.html" rel="nofollow">http://mywebsite.com/page.html</a>  . I want to store that link A was obtained through this webpage. So after I crawl , I can remove pages and obtain only Type A pages and I have that, this type A webpage was obtained from what particular webpage .Can I configure nutch for doing so ? , If Yes , Some hints . Do I need to modify source code. Write my own plugin in nutch . ?"</p>