title: How to scrape website while iterate on multiple pages
body: <p>Trying to scrape this website using python beautifulsoup:
  <a href="https://www.leandjaya.com/katalog" rel="nofollow noreferrer">https://www.leandjaya.com/katalog</a></p>
  <p>having some challenges in navigating the multiple pages of the website and scrape it
  using python, this website has 11 pages, and curious to know the best option to
  achieve this like use for loop and will break the loop if the page doesnt exist.</p>
  <pre><code>this is my initial code, I have set a big number 50, however seems this is not a good option.
  page = 1
  while page != 50:
      url=f&quot;https://www.leandjaya.com/katalog/ss/1/{page}/&quot;
      main = requests.get(url)
      pmain = BeautifulSoup(main.text,'lxml')
      page = page + 1
  Sample output:
  https://www.leandjaya.com/katalog/ss/1/1/
  https://www.leandjaya.com/katalog/ss/1/2/
  https://www.leandjaya.com/katalog/ss/1/3/
  https://www.leandjaya.com/katalog/ss/1/&lt;49&gt;/
  </code></pre>