{"action":"closed","issue":{"url":"https://api.github.com/repos/scrapy/scrapy/issues/1931","repository_url":"https://api.github.com/repos/scrapy/scrapy","labels_url":"https://api.github.com/repos/scrapy/scrapy/issues/1931/labels{/name}","comments_url":"https://api.github.com/repos/scrapy/scrapy/issues/1931/comments","events_url":"https://api.github.com/repos/scrapy/scrapy/issues/1931/events","html_url":"https://github.com/scrapy/scrapy/issues/1931","id":148182712,"number":1931,"title":"start_requests() not taking the Dynamic Variables","user":{"login":"kgrvamsi","id":1199031,"avatar_url":"https://avatars.githubusercontent.com/u/1199031?v=3","gravatar_id":"","url":"https://api.github.com/users/kgrvamsi","html_url":"https://github.com/kgrvamsi","followers_url":"https://api.github.com/users/kgrvamsi/followers","following_url":"https://api.github.com/users/kgrvamsi/following{/other_user}","gists_url":"https://api.github.com/users/kgrvamsi/gists{/gist_id}","starred_url":"https://api.github.com/users/kgrvamsi/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kgrvamsi/subscriptions","organizations_url":"https://api.github.com/users/kgrvamsi/orgs","repos_url":"https://api.github.com/users/kgrvamsi/repos","events_url":"https://api.github.com/users/kgrvamsi/events{/privacy}","received_events_url":"https://api.github.com/users/kgrvamsi/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-04-13T21:03:11Z","updated_at":"2016-04-13T21:47:56Z","closed_at":"2016-04-13T21:47:56Z","body":"I'm being trying to iterate a loop of urls in which i have three layers of scraping done\r\n\r\n1)Scrapy the first URL and get the respective data\r\n2)Based upon the data use that in one of the for loop range\r\n3)Run the Scraping and fetch new urls and append it to a variable and then again parse them\r\n\r\nHere is the example code that i'm looking out for\r\n\r\n```\r\nfrom scrapy.spider import BaseSpider\r\nfrom scrapy.http import Request\r\n\r\nclass TestSpider(BaseSpider):\r\n    name = \"test_spider\"\r\n    allowed_domains=[\"amazon.com\"]\r\n    some_value=0\r\n    data=[]\r\n    def start_requests(self):\r\n        yield Request(first-url,callback=self.parse)\r\n       ''' this is the place the some_value is not replaced with the new value '''\r\n        for i in range(2, some_value):\r\n            url = 'http://www.amazon.com/dp/%i/' % i\r\n            yield Request(url,callback=self.somefunction)\r\n        \r\n         for site in data:\r\n           yield Request(site,callback=self.somefunction)\r\n\r\n    def parse(self, response):\r\n        '''does nothing'''\r\n        if self.some_value == 0:\r\n              self.some_value = int(response.xpath(somexpath).extract())\r\n        url =  response.xpath(somexpath).extract()[0]\r\n        self.data.append(str(url))\r\n        print 'parse'\r\n         \r\n         "}}