title: How to go to next page when web crawling using Python
body: <p>I trying to do web crawling using python. But I can't figure out how to change pages automatically. 
  So I found the pattern but I don't know how to go to next page automatically until it reaches end of page. </p>
  
  <p>so the pattern is </p>
  
  <p>'http//.../sortBy=helpful&amp;pageNumber=0'</p>
  
  <p>'http//.../sortBy=helpful&amp;pageNumber=1'</p>
  
  <p>'http//.../sortBy=helpful&amp;pageNumber=2'</p>
  
  <p>'http//.../sortBy=helpful&amp;pageNumber=3'<br>
  and so on ...</p>
  
  <pre><code>import re
  from urllib.parse import urljoin
  
      def review_next_page(page=1):
             list_url = 'https://www.amazon.com/Quest-Nutrition-Protein-Apple-2-12oz/product-reviews/B00U3RGAMW/ref=cm_cr_arp_d_paging_btm_2?ie=UTF8&amp;showViewpoints=1&amp;sortBy=recent&amp;pageNumber={0}'.format(page)
             list_url = [urljoin(list_url, review_link) for review_link in ???]
             return list_url  
  </code></pre>
  
  <p>I am trying to change last number increases by 1 until it reaches the end...
  Should I use for loop?</p>
  
  <p>Thanks in advance!</p>