title: Scraping a dynamic URL that changes based on time using Selenium in Python
body: <p>I am attempting to scrape the following URL:</p>
  
  <p><a href="https://www.oddsportal.com/soccer/england/premier-league/liverpool-norwich-4IMoMG3q/" rel="nofollow noreferrer">https://www.oddsportal.com/soccer/england/premier-league/liverpool-norwich-4IMoMG3q/</a></p>
  
  <p>Using the Network tab in Chrome's developer tools, you can see there is an API that feeds the data into the website - which is in the form of a JSON - and looks as below. This data is what I am trying to scrape.</p>
  
  <p><a href="https://fb.oddsportal.com/feed/match/1-1-4IMoMG3q-5-2-yj1e3.dat?_=1562831112277" rel="nofollow noreferrer">https://fb.oddsportal.com/feed/match/1-1-4IMoMG3q-5-2-yj1e3.dat?_=1562831112277</a></p>
  
  <p>This is the code I am currently trying to scrape this with:</p>
  
  <pre><code>from selenium.webdriver.chrome.options import Options
  import json
  import urllib.parse
  from time import time
  
  options = Options()
  options.headless = True
  driver = webdriver.Chrome(options=options)
  
  # Access the initial webpage to create the info_dict (including the match_id, and hash)
  driver.get('https://www.oddsportal.com/soccer/england/premier-league/liverpool-norwich-4IMoMG3q')
  page = driver.page_source
  info_dict = json.loads(page.split('var page = new PageEvent(')[-1].split(');')[0])
  xhash = urllib.parse.unquote(info_dict['xhash'])
  match_id = info_dict['id']
  
  # Access to the feed URL based on the values from the info_dict
  driver.get('http://fb.oddsportal.com/feed/match/1-1-{}-1-2-{}.dat?_={}'.format(match_id, xhash, int(round(time()*1000)) + 1000))
  print(driver.page_source)
  </code></pre>
  
  <p>The URL is built up of 3 factors - the match_id, the hash, and epoch time in milliseconds. However, when I try to access this in Selenium, I get the following response:</p>
  
  <p><code>globals.jsonpCallback('/feed/match/1-1-4IMoMG3q-1-2-yjb3a.dat?_=1562795864899', {'e':'404'});</code></p>
  
  <p>Would really appreciate any help with this, as I don't really understand where I'm going wrong!</p>