title: Scrapy do something before resume
body: <p>I'm using scrapy to crawl a web-site with authentication.<br>
  I want to be able to save the state of the crawler and I use </p>
  
  <pre><code>scrapy crawl myspider -s JOBDIR=mydir
  </code></pre>
  
  <p>After I resume with the same command I want to be able to login to the website <strong>before</strong> it <strong>reschedules</strong> all saved requests.   </p>
  
  <p>Basically, I want to be sure that my function <code>login()</code> and <code>after_login()</code> will be called before any other request is scheduled and executed. And I don't want to use cookies, because they don't allow me to pause the crawling for a long time.<br>
  I can call <code>login()</code> in <code>start_requests()</code>, but this works only when I run the crawler for the first time.</p>
  
  <pre class="lang-py prettyprint-override"><code>class MyCrawlSpider(CrawlSpider):
      # ...
  
      START_URLS = ['someurl.com', 'someurl2.com']
      LOGIN_PAGE = u'https://login_page.php'
  
      def login(self):
          return Request(url=self.LOGIN_PAGE, callback=self.login_email,
                        dont_filter=True, priority=9999)    
  
      def login_form(self, response):
          return FormRequest.from_response(response,
                                            formdata={'Email': 'myemail',
                                                      'Passwd': 'mypasswd'},
                                            callback=self.after_login, 
                                            dont_filter=True,
                                            priority=9999)
  
      def after_login(self, response):
          if "authentication failed" in response.body:
              self.logger.error("Login failed")
              return
          else:
              print("Login Successful!!")
              self.is_logged_in = True
              for url in self.START_URLS:
                  yield Request(url, callback=self.parse_artists_json, dont_filter=True)
  </code></pre>
  
  <p><strong>Bottomline:</strong> Is there any callback which will always be called when I resume crawling with <code>-s JOBDIR=...</code> option before rescheduling previous requests? I will use it to call <code>login()</code> method.</p>