title: Creating a dataframe with scraped data
body: <p>I am trying to scraping data as follows:</p>
  <pre><code>import requests
  from bs4 import BeautifulSoup
  from concurrent.futures import ThreadPoolExecutor
  import pandas as pd
  
  daterange = pd.date_range('02-25-2015', '09-16-2020', freq='D')
  
  def main(req, date):
      r = req.get(f&quot;https://it.sputniknews.com/politica/{date.strftime('%Y%m%d')}&quot;)
      print(r, r.content)
      soup = BeautifulSoup(r.content, 'html.parser')
      tag=None
      print (soup.select(&quot;b-plainlist&quot;))
      #for tag in soup.select(&quot;.b-plainlist &quot;):
          #print(tag.select_one(&quot;.b-plainlist__date&quot;).text)
          #print(tag.select_one(&quot;.b-plainlist__title&quot;).text)
          #print(tag.find_next(class_=&quot;b-plainlist__announce&quot;).text.strip())
      
      return tag.select_one(&quot;.b-plainlist__date&quot;).text, tag.select_one(&quot;.b-plainlist__title&quot;).text, tag.find_next(class_=&quot;b-plainlist__announce&quot;).text.strip()
  
  
  with ThreadPoolExecutor(max_workers=30) as executor:
      with requests.Session() as req:
          fs = [executor.submit(main, req, date) for date in daterange]
          allin = []
          for f in fs:
              allin.append(f.result())
          df = pd.DataFrame.from_records(
              allin, columns=[&quot;Date&quot;, &quot;Title&quot;, &quot;Content&quot;])
          print(df)
  </code></pre>
  <p>trying to create a dataframe with the Date, Title and Content though time.</p>
  <p>This code should be ok, but I cannot create a 'clean' dataframe so I think there is something wrong in the tags.
  Can you please have a look? thanks</p>