title: why does my webscraper fail to scrape data from Puma website?
body: <p>I am writing a script to scrape product details from Puma's official website- <a href="https://in.puma.com/in/en/home" rel="nofollow noreferrer">https://in.puma.com/in/en/home</a></p>
  <p>The script runs without any errors as such. However, it just does not scrape details and returns empty values. I was previously working with xpaths but I made changes to my code and have taken class names for scraping data like product price,title,colors.
  here's my code:</p>
  <pre><code>def puma_product_scraper():
      titles = []
      prices = []
      p_colors = []
      options = webdriver.ChromeOptions()
      options.add_argument('start-maximized')
      options.add_argument(&quot;window-size=1200x600&quot;)
      options.add_experimental_option(&quot;excludeSwitches&quot;, [&quot;enable-automation&quot;])
      options.add_experimental_option('useAutomationExtension', False)
  
      browser = webdriver.Chrome(ChromeDriverManager().install(), options=options)
      browser.get(&quot;https://in.puma.com/in/en/search?q=running&amp;originalphrase=running&quot;)
      browser.implicitly_wait(10)
  
      # Begin scraping elements
  
      # browser.find_element_by_class_name('btn').click()
      product_cards_container = browser.find_element(By.CLASS_NAME, &quot;product-grid-container&quot;)
      product_cards = product_cards_container.find_elements(By.CLASS_NAME, &quot;col-lg-3 col-sm-4 col-md-4 col-6&quot;)
      for card in product_cards:
          title = card.find_element(By.CLASS_NAME, &quot;product-tile-info-text&quot;)
          colors = card.find_element(By.CLASS_NAME, &quot;product-tile-swatch-container&quot;)
          price = card.find_element(By.CLASS_NAME, &quot;product-tile-price-new product-tile__price--new&quot;)
          titles.append(title.text)
          p_colors.append(colors.text)
          prices.append(price.text)
  
      # Storing scraped content
      df = pd.DataFrame({'Product Name': titles, 'Price': prices, 'Colors': p_colors})
      df.to_csv('PUMA_Scraping.csv', index=False)
      df.head()
      print('Total items related to running on Puma:', len(titles))
  </code></pre>
  <p>Here's the output that I get:</p>
  <pre><code>Total items related to running on Puma: 0
  </code></pre>
  <p>If there are problems related to class names, then It shouldve displayed an error message, but I have not recieved any messages as such? Where is the problem exactly? please help.</p>
  <p><strong>EDIT</strong>:
  As @Joran suggested applied variations for find_element.</p>
  <pre><code>wait = WebDriverWait(browser, 10)
      # browser.find_element_by_class_name('btn').click()
      product_cards_container = wait.until(EC.presence_of_element_located((By.CLASS_NAME, &quot;product-grid-container&quot;)))
  </code></pre>
  <p>still did not work.</p>