title: Data Scraping from Vivino.com - wine information and reviews
body: <p>In order to write my master thesis, I need to collect data. Now, I would like to collect data from Vivino.com, but I don't have any experience in web scraping. I already saw some questions about this, but I would like to collect all the information about the wines (name, country, rating, description, price etc.) and the reviews of the wines.</p>
  <pre><code>import requests
  import pandas as pd
  
  r = requests.get(
      &quot;https://www.vivino.com/api/explore/explore&quot;,
      params = {
          &quot;country_code&quot;: &quot;FR&quot;,
          &quot;country_codes[]&quot;:&quot;pt&quot;,
          &quot;currency_code&quot;:&quot;EUR&quot;,
          &quot;grape_filter&quot;:&quot;varietal&quot;,
          &quot;min_rating&quot;:&quot;1&quot;,
          &quot;order_by&quot;:&quot;price&quot;,
          &quot;order&quot;:&quot;asc&quot;,
          &quot;page&quot;: 1,
          &quot;price_range_max&quot;:&quot;500&quot;,
          &quot;price_range_min&quot;:&quot;0&quot;,
          &quot;wine_type_ids[]&quot;:&quot;1&quot;
      },
      headers= {
          &quot;User-Agent&quot;: &quot;Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0&quot;
      }
  )
  results = [
      (
          t[&quot;vintage&quot;][&quot;wine&quot;][&quot;winery&quot;][&quot;name&quot;], 
          f'{t[&quot;vintage&quot;][&quot;wine&quot;][&quot;name&quot;]} {t[&quot;vintage&quot;][&quot;year&quot;]}',
          t[&quot;vintage&quot;][&quot;statistics&quot;][&quot;ratings_average&quot;],
          t[&quot;vintage&quot;][&quot;statistics&quot;][&quot;ratings_count&quot;]
      )
      for t in r.json()[&quot;explore_vintage&quot;][&quot;matches&quot;]
  ]
  dataframe = pd.DataFrame(results,columns=['Winery','Wine','Rating','num_review'])
  
  print(dataframe)
  </code></pre>
  <p>With this code, I can collect ['Winery' 'Wine' 'Rating' 'num_review']</p>
  <p>And with the following code I can collect reviews:</p>
  <pre><code>import re
  import json
  import requests
  
  headers = {
      &quot;User-Agent&quot;: &quot;Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:89.0) Gecko/20100101 Firefox/89.0&quot;,
  }
  
  
  url = &quot;https://www.vivino.com/FR/en/dauprat-pauillac/w/3823873?year=2017&amp;price_id=24797287&quot;
  api_url = (
      &quot;https://www.vivino.com/api/wines/{id}/reviews?per_page=9999&amp;year={year}&quot;
  ) # &lt;-- increased the number of reviews to 9999
  
  id_ = re.search(r&quot;/(\d{5,})&quot;, url).group(1)
  year = re.search(r&quot;year=(\d+)&quot;, url).group(1)
  
  data = requests.get(api_url.format(id=id_, year=year), headers=headers).json()
  
  # uncomment this to print all data:
  # print(json.dumps(data, indent=4))
  
  for r in data[&quot;reviews&quot;]:
      print(r[&quot;note&quot;])
      print(&quot;-&quot; * 80)
  </code></pre>
  <p>Can someone please help me how I can combine all this information? So, all the wine information including corresponding reviews?</p>
  <p>Thank you in advance!!</p>