title: Scraping multiple pages with Python Beautifulsoup -- only returning data from last page
body: <p>I am trying to loop through multiple pages to scrape data with Python and Beautifulsoup. My script works for one page, but when trying to iterate through multiple pages, it only returns the data from the last page scraped. I think there may be something wrong in the way I am looping or storing/appending the <code>player_data</code> list.</p>
  
  <p>Here is what I have thus far -- any help is much appreciated.</p>
  
  <pre><code>#! python3
  # downloadRecruits.py - Downloads espn college basketball recruiting database info
  
  import requests, os, bs4, csv
  import pandas as pd
  
  # Starting url (class of 2007)
  base_url = 'http://www.espn.com/college-sports/basketball/recruiting/databaseresults/_/class/2007/page/'
  
  # Number of pages to scrape (Not inclusive, so number + 1)
  pages = map(str, range(1,3))
  
  # url for starting page
  url = base_url + pages[0]
  
  for n in pages:
      # Create url
      url = base_url + n
  
      # Parse data using BS
      print('Downloading page %s...' % url)
      res = requests.get(url)
      res.raise_for_status()
  
      # Creating bs object
      soup = bs4.BeautifulSoup(res.text, "html.parser")
  
      table = soup.find('table')
  
      # Get the data
      data_rows = soup.findAll('tr')[1:]
  
      player_data = []
      for tr in data_rows:
          tdata = []
          for td in tr:
              tdata.append(td.getText())
  
              if td.div and td.div['class'][0] == 'school-logo':
                  tdata.append(td.div.a['href'])
  
          player_data.append(tdata)
  
  print(player_data)
  </code></pre>