title: How to speed up scraping in python?
body: <p>i want to display 10 hotels which are best from user perspective. Suppose user will enter 'pool' then i have to math the keyword 'pool' in the user reviews from tripadvisor then take a count and display the top 10 hotels name according to count. For this purpose i am currently scrapping all the reviews of hotels(dubai) then i will match keyword and display the top 10 hotel names.but hotel review scrapping is taking too much time what i can do? any help? ANy other method other than scraping?this is my code for scrapping reviews :</p>
  
  <pre><code>import requests
  from bs4 import BeautifulSoup
  
  offset = 0
  url = 'https://www.tripadvisor.com/Hotels-g295424-oa' + str(offset) + '-Dubai_Emirate_of_Dubai-Hotels.html#EATERY_LIST_CONTENTS'
  
  urls = []
  r = requests.get(url)
  soup = BeautifulSoup(r.text, "html.parser")
  
  for link in soup.find_all('a', {'last'}):
        page_number = link.get('data-page-number')
        last_offset = int(page_number) * 30
        print('last offset:', last_offset)
  
  for offset in range(0, last_offset, 30):
     print('--- page offset:', offset, '---')
  
       url = 'https://www.tripadvisor.com/Hotels-g295424-oa' + str(offset) + '-Dubai_Emirate_of_Dubai-Hotels.html#EATERY_LIST_CONTENTS'
  
      r = requests.get(url)
      soup = BeautifulSoup(r.text, "html.parser")
  
      for link in soup.find_all('a', {'property_title'}):
          iurl='https://www.tripadvisor.com/' + link.get('href')
  
          r = requests.get(iurl)
          soup = BeautifulSoup(r.content, "lxml")
          #look for the partial entry of the review
          resultsoup = soup.find_all("p", {"class" : "partial_entry"})
  
             for review in resultsoup:
                review_list = review.get_text()
                print(review_list)
  </code></pre>