title: Scrapy LinkExtractor - which RegEx to follow?
body: <p>I'm trying to scrape a category from amazon but the links that I get in Scrapy are different from the ones in the browser. Now I am trying to follow the next page trail and in Scrapy (printed <code>response.body</code> into a txt file) I see those links:</p>
  
  <pre><code>&lt;span class="pagnMore"&gt;...&lt;/span&gt;
  &lt;span class="pagnLink"&gt;&lt;a href="/s?ie=UTF8&amp;page=4&amp;rh=n%3A2619533011%2Ck%3Apet%20supplies%2Cp_72%3A2661618011%2Cp_n_date_first_available_absolute%3A2661609011" &gt;4&lt;/a&gt;&lt;/span&gt;
  &lt;span class="pagnCur"&gt;5&lt;/span&gt;
  &lt;span class="pagnLink"&gt;&lt;a href="/s?ie=UTF8&amp;page=6&amp;rh=n%3A2619533011%2Ck%3Apet%20supplies%2Cp_72%3A2661618011%2Cp_n_date_first_available_absolute%3A2661609011" &gt;6&lt;/a&gt;&lt;/span&gt;
  &lt;span class="pagnMore"&gt;...&lt;/span&gt;
  &lt;span class="pagnDisabled"&gt;20&lt;/span&gt;
  &lt;span class="pagnRA"&gt; &lt;a title="Next Page"
                     id="pagnNextLink"
                     class="pagnNext"
                     href="/s?ie=UTF8&amp;page=6&amp;rh=n%3A2619533011%2Ck%3Apet%20supplies%2Cp_72%3A2661618011%2Cp_n_date_first_available_absolute%3A2661609011"&gt;
  &lt;span id="pagnNextString"&gt;Next Page&lt;/span&gt;
  </code></pre>
  
  <p>I'd like to follow the pagnNextString link, but my spider doesn't even start crawling:</p>
  
  <pre><code>Rule(SgmlLinkExtractor(allow=("n\%3A2619533011\%", ),restrict_xpaths=('//*[@id="pagnNextLink"]',)) , callback="parse_items", follow= True),
  </code></pre>
  
  <p>If I get rid of the rule or do sth. like <code>'^http.*'</code> it's working but it follows everything.
  What am I doing wrong here?</p>