title: I use crawlspider how can I stop redirect in rule
body: <p>this is my rule, this is my first use crawlspider, so how can I stop redirect(302) in my rule</p>
  
  <pre><code>rules = (
          Rule(LinkExtractor(allow=r'zhaopin/.*'), follow=True),
          Rule(LinkExtractor(allow=r'gongsi/j.*/.html'), follow=True),
          Rule(LinkExtractor(allow=r'jobs/.*.html'), callback='parse_job', follow=True),
      )
  </code></pre>
  
  <p>this is the debug,you can see, </p>
  
  <pre><code>2017-07-05 09:20:24 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/CTO/&gt;
  2017-07-05 09:20:25 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/jiagoushi/&gt;
  2017-07-05 09:20:25 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/C%23/&gt;
  2017-07-05 09:20:25 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/youxizhizuoren/&gt;
  2017-07-05 09:20:25 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/chanpinbujingli/&gt;
  2017-07-05 09:20:25 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/wuxianchanpinshejishi/&gt;
  2017-07-05 09:20:25 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/wangyechanpinshejishi/&gt;
  2017-07-05 09:20:25 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/chanpinshixisheng/&gt;
  2017-07-05 09:20:25 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/dbaqita/&gt;
  2017-07-05 09:20:25 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to &lt;GET https://passport.lagou.com/login/login.html?msg=validation&amp;uStatus=2&amp;clientIp=60.211.222.66&gt; from &lt;GET https://www.lagou.com/zhaopin/guanggaoshejishi/&gt;
  2017-07-05 09:20:26 [scrapy.crawler] INFO: Received SIG_UNBLOCK, shutting down gracefully. Send again to force 
  </code></pre>