title: How to stop robots crawling pagination using robots.txt?
body: <p>I have various paginations on my site and I want to stop google and other search engines crawling the index of my paginations.</p>
  
  <p>Example of a crawled page:</p>
  
  <pre><code>http://www.mydomain.com/explore/recently-updated/index/12
  </code></pre>
  
  <p>How can I, using robots.txt deny robots from crawling any pages that contain /index/ ?</p>