title: How to get all of data if tags with a lots of nested?
body: <p>I try to get first day movie data(today is 8/25), and find the href from website is</p>
  <p><code>https://www.vscinemas.com.tw/vsweb/theater/detail.aspx?id=16#movieTime-1508017172</code></p>
  <p>I want to get all of the movie data (name, gradle, time, type...etc), for example:</p>
  <pre><code>FAST AND FURIOUS 9
  childview
  數位
  ['11:30', '12:45', '14:25', '15:30', '17:15', '18:15', '20:00', '21:00', '22:45', '23:45']
  
  1
  IMAX
  ['12:00', '16:45', '21:30']
  
  2
  ['GC 數位']
  ['13:00', '16:10', '18:00', '20:50', '23:35']
  </code></pre>
  <p>But I the website tags rule, if some movies have multiple types like IMAX 4DX, it will nested another <code>&lt;div class=&quot;movieDay&quot; /&gt;</code>, and other movies data will under the <code>&lt;div /&gt;</code>, of course, if still has another multiple types of movies, it will nested again!</p>
  <p>When I write until inThreeNode, I found it is not right, because I have to check and nested over and over again. It must have some ways to check the rule and write the more clear code, but I have no idea how to achieve it.</p>
  <p>Here is my spider, any helps would be appreciated.</p>
  <pre><code># -*- coding: utf-8 -*-
  import scrapy
  from selenium import webdriver
  from selenium.webdriver.chrome.options import Options
  import sys
  import time
  
  class TesttwoSpider(scrapy.Spider):
      name = 'testtwo'
      allowed_domains = ['www.vscinemas.com.tw/vsweb/']
      start_urls = ['https://www.vscinemas.com.tw/vsweb/theater/detail.aspx?id=16']
  
      # print(sys.path)
  
      # use selenium
      # options = Options()
      # options.add_argument(&quot;--disable-notifications&quot;)
      
      # chrome = webdriver.Chrome('/Users/motogod19/chromedriver', chrome_options=options)
      # chrome.get(&quot;https://www.facebook.com/&quot;)
  
      def parse(self, response):
          print('************ Start parse ************')
          # childview one of movie types
          # teenager one of movie types
  
          firstHtmlNodes = response.xpath('//div[@class=&quot;theaterTime&quot;]/article[@id=&quot;movieTime-1508017172&quot;]')
          for firstHtmlNode in firstHtmlNodes:
              cnName = firstHtmlNode.xpath('./h2/text()').extract()
  
              print('Check length')
              print(len(cnName))
  
              getFistFloorEnNameLength = len(cnName)
              for index in range(len(cnName)):
                  movieName = firstHtmlNode.xpath('./h2[{}]/text()'.format(index+1)).extract_first()
                  movieGradle = firstHtmlNode.xpath('./h1[{}]/span/@class'.format(index+1)).extract_first()
                  movieType = firstHtmlNode.xpath('./div[{}]/h4/text()'.format(index+1)).extract_first()
                  movieTime = firstHtmlNode.xpath('./div[{}]/ul/li/a/text()'.format(index+1)).extract()
                  print(movieName)
                  print(movieGradle)
                  print(movieType)
                  print(movieTime)
                  # lats one check &lt;div class=&quot;movieDay&quot; /&gt; exists or not
                  if getFistFloorEnNameLength == index + 1:
                      nextNode = firstHtmlNode.xpath('./div[{}]/div[1]'.format(index+1))
                      if len(nextNode) != 0:
                          movieType = nextNode.xpath('./h4/text()').extract_first()
                          movieTime = nextNode.xpath('./ul/li/a/text()').extract()
                          print('1')
                          print(movieType)
                          print(movieTime)
                          
                          inTwoNode = nextNode.xpath('./div[1]')
  
                          if len(inTwoNode) != 0:
                              movieType = inTwoNode.xpath('./h4/text()').extract()
                              movieTime = inTwoNode.xpath('./ul/li/a/text()').extract()
                              print('2')
                              print(movieType)
                              print(movieTime)
                              # I stop here, because I have to check and nested over and over again.
                              inThreeNode = inTwoNode.xpath('./h2/text()')
  
                          else:
                              movieName = firstHtmlNode.xpath('./div[{}]/h2/text()'.format(index+1))
                              print('3')
                              print(movieName)
  </code></pre>