title: Crawling authentication based pages using apache nutch
body: <p>How can I crawl authentication based pages using nutch? I have done all the required settings in nutch-site.xml, nutch-default.xml and httpclient-auth.xml. Still it shows the following:</p>
  
  <pre><code>Generator: 0 records selected for fetching, exiting ...
  Stopping at depth=1 - no more URLs to fetch.
  </code></pre>
  
  <p>I have followed the the following links
  <a href="https://stackoverflow.com/questions/25183951/crawling-websites-which-ask-for-authentication">link 1</a>,
  <a href="http://gingercart.com/Home/search-and-crawl/nutch-custom-authentication-cookies-session-management-to-crawl-secure-enterprise-websites" rel="nofollow noreferrer">link 2</a>. But still my crawler doesn't crawl the pages. Is there any way in which I can use an API key to help crawling?</p>