title: How to scrape multiple URLs effectively in BS4
body: <p>I am trying to find an effective way to scrape multiple pages in BS4. I am able to scrape the first page easily and get all of the data I need but unfortunately, not all data is on it. There are 2 other pages to scrape and rather than hard coding this and changing the URL for the second and third pages I was wondering if there was a more elegant way of doing this in Python using BS4. The only part of the URL that needs to be changed is page=1 to the respective page number(either 1,2,3).  </p>
  
  <pre><code>import csv 
  import requests
  from bs4 import BeautifulSoup
  
  
  url = "https://www.congress.gov/members?q={%22congress%22:%22115%22}&amp;pageSize=250&amp;page=1"
  
  headers = {'User-Agent': 'Mozilla/5.0'}
  response = requests.get(url, headers=headers)
  soup = BeautifulSoup(response.text, "html.parser")
  
  names = soup.find_all()
  
  items = soup.find_all("li","expanded")
  for item in items:
      print(item.text)
      print(item.find("a"))
      with open('web.csv', 'a') as csv_file:
          writer = csv.writer(csv_file)
          writer.writerow([item.find("a").encode('utf-8')])
  </code></pre>