title: How to accumulate in a df parsed data through a loop with pandas from a web scrapping?
body: <p>I want to create a df with an historical dataset by scrapping a website, but I struggle to accumulate the full period within the loop. I am able to download a day, but when I try to create a loop to storage a set of iterations I am not able to accumulate the data in the dataframe.</p>
  <p>The df I want to create from the <code>start_date</code> to the <code>end_date</code> is as follows:</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th>Fecha</th>
  <th>PeríodeTU</th>
  <th>TM°C</th>
  <th>HRM%</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td>single_date</td>
  <td></td>
  <td></td>
  <td></td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>Where <strong>Fecha</strong> is a result of adding a columns with the <code>single_date</code> of the code below, and the rest of the columns are actual data from the website scrapped.</p>
  <p>I have tried this:</p>
  <pre><code>def daterange(start_date, end_date):
      for n in range(int ((end_date - start_date).days)):
          yield start_date + timedelta(n)
  
  start_date = date(2020, 6, 1)
  end_date = date(2021, 3, 3)
  
  
  for single_date in daterange(start_date, end_date):
      #URL API Meteo.cat con la fecha
      url = &quot;https://www.meteo.cat/observacions/xema/dades?codi=V3&amp;dia=&quot;+str(single_date)+&quot;T00:00Z&quot;        
  
      # GET a la API
      res = requests.get(url)
      soup = BeautifulSoup(res.content,'lxml')
      table = soup.find_all('table')[2]
      df_table = pd.read_html(str(table))[0]
      df_table['Fecha'] = single_date
  
  
  data['Fecha'] = df['Fecha']
  data['Hora'] = df['PeríodeTU']
  data['Temperatura_Media'] = df['TM°C']
  data['Humedad_Relativa'] = df['HRM%']
  data.to_csv('Data/tempset.csv', header=True, index=False)
  </code></pre>
  <p><code>df_table</code> only saves the last date, and I want to save the full period iterated.</p>
  <p><em><strong>Does anyone know how to deal with this situation?</strong></em></p>