title: Extract data with infinite scrolling Ajax with Scrapy
body: <p>I am a beginner in python and scrapy.
  I need help, I need to extract a product listing but he site hast a "view more" products at the end that execute with ajax an text/html request and load another html with new products.</p>
  
  <pre><code>import scrapy
  from scrapy.http import Request
  
  
  class ProdSpider(scrapy.Spider):
  name = "easy"
  allowed_domains = ["easy.com.ar"]
  start_urls = ["https://www.easy.com.ar/webapp/wcs/stores/servlet/es/easyar/search/AjaxCatalogSearchResultContentView?searchTermScope=&amp;searchType=1002&amp;filterTerm=&amp;orderBy=&amp;maxPrice=&amp;showResultsPage=true&amp;langId=-5&amp;sType=SimpleSearch&amp;metaData=&amp;pageSize=12&amp;manufacturer=&amp;resultCatEntryType=&amp;catalogId=10051&amp;pageView=image&amp;searchTerm=&amp;minPrice=&amp;categoryId=39652&amp;storeId=10151&amp;beginIndex=12"]
  beginIndex_index = 12
  
  def parse(self, response):
      SECTION_SELECTOR = '.thumb-product'
      for soar in response.css(SECTION_SELECTOR):
  
               Link = 'div.dojoDndItem a ::attr(href)'
               # Marca = 'p.brand a ::text'
               Nombre = 'div.thumb-name a ::text'
               # Link = 'p.brand a ::attr(href)'
               # SKU = './/p[@class="sku"]/text()' #p.sku ::text'
               Price = './/span[@id="tarj-mas-edit"]/text()' #["0"].parentElement.innerText    .//span[@class="thumb-price-e"]/text()
  
               yield {
                 'Link': soar.css(Link).extract_first(),
              # 'Marca': soar.css(Marca).extract_first(),
                 'Nombre': soar.css(Nombre).re_first(r'\n\s*(.*)'), # Limpia espacios y caracteres especiales
              # 'Link': soar.css(Link).extract_first(),   
              # 'SKU': soar.xpath(SKU).re_first(r'SKU:\s*(.*)'),
                 'Price': soar.xpath(Price).re_first(r'\n\s*(.*)'),
                }
      # here if no products are available , simply return, means exiting from
      # parse and ultimately stops the spider
  
  
      self.beginIndex_index += 12
      if beginIndex_index:
          yield Request(url="https://www.easy.com.ar/webapp/wcs/stores/servlet/es/easyar/search/AjaxCatalogSearchResultContentView?searchTermScope=&amp;searchType=1002&amp;filterTerm=&amp;orderBy=&amp;maxPrice=&amp;showResultsPage=true&amp;langId=-5&amp;sType=SimpleSearch&amp;metaData=&amp;pageSize=12&amp;manufacturer=&amp;resultCatEntryType=&amp;catalogId=10051&amp;pageView=image&amp;searchTerm=&amp;minPrice=&amp;categoryId=39652&amp;storeId=10151&amp;beginIndex=%s" % (self.beginIndex_index + 12),
                        callback=self.parse)
  </code></pre>
  
  <p>I tried with code above but only 12 products are getting captured. The only parameter that changes in url is "beginIndex=12", Y want to sum +12 to the url until the products listings over. I'm getting stuck with this issue! </p>
  
  <p>Thanks! </p>