title: Scrapy : create folder structure out of downloaded images based on the url from which images are being downloaded
body: <p>I have an array of links that define the structure of a website. While downloading images from these links, I want to simultaneously place the downloaded images in a folder structure similar to the website structure, and not just rename it (as answered in <a href="https://stackoverflow.com/questions/6194041/scrapy-image-download-how-to-use-custom-filename">Scrapy image download how to use custom filename</a>)</p>
  
  <p>My code for the same is like this:</p>
  
  <pre><code>class MyImagesPipeline(ImagesPipeline):
      """Custom image pipeline to rename images as they are being downloaded"""
      page_url=None
      def image_key(self, url):
          page_url=self.page_url
          image_guid = url.split('/')[-1]
          return '%s/%s/%s' % (page_url,image_guid.split('_')[0],image_guid)
  
      def get_media_requests(self, item, info):
          #http://store.abc.com/b/n/s/m
          os.system('mkdir '+item['sku'][0].encode('ascii','ignore'))
          self.page_url = urlparse(item['start_url']).path #I store the parent page's url in start_url Field
          for image_url in item['image_urls']:
              yield Request(image_url)
  </code></pre>
  
  <p>It creates the required folder structure but when I go into the folders in deapth, I see that the files have been misplaced in the folders.</p>
  
  <p>I'm suspecting that it is happening because the "get_media_requests" and "image_key" functions might be executing asynchronously hence the value of "page_url" changes before it is used by the "image_key" function.</p>