title: Downloader Middleware to ignore all requests to a certain URL in scrapy
body: <p>I am trying to define a custom downloader middleware in Scrapy to ignore all requests to a particular URL (these requests are redirected from other URLs, so I can't filter them out when I generate the requests in the first place).</p>
  
  <p>I have the following code, the idea of which is to catch this at the response processing stage (as I'm not exactly sure how requests redirecting to other requests works), check the URL, and if it matches the one I'm trying to filter out then return an IgnoreRequest exception, if not, return the response as usual so that it can continue to be processed.</p>
  
  <pre><code>from scrapy.exceptions import IgnoreRequest
  from scrapy import log
  
  class CustomDownloaderMiddleware:
  
      def process_response(request, response, spider):
          log.msg("In Middleware " + response.url, level=log.WARNING)
          if response.url == "http://www.achurchnearyou.com//":
              return IgnoreRequest()
          else:
              return response
  </code></pre>
  
  <p>and I add this to the dict of middlewares:</p>
  
  <pre><code>DOWNLOADER_MIDDLEWARES = {
      'acny.middlewares.CustomDownloaderMiddleware': 650
  }
  </code></pre>
  
  <p>with a value of 650, which should - I think - make it run directly after the RedirectMiddleware.</p>
  
  <p>However, when I run the crawler, I get an error saying:</p>
  
  <pre><code>ERROR: Error downloading &lt;GET http://www.achurchnearyou.com/venue.php?V=00001&gt;: process_response() got multiple values for keyword argument 'request'
  </code></pre>
  
  <p>This error is occurring on the very first page crawled, and I can't work out why it is occurring - I think I've followed what the manual said to do. What am I doing wrong?</p>