title: prepare scrapy data for normalized postgresql database
body: <p>I have build a postgresql data model to store football results. So I have the following tables with foreign keys linking them.</p>
  
  <p>country table:
  id,country</p>
  
  <p>Club table:
  id,id_country,club</p>
  
  <p>Matches table:
  gamedate,id_club_home,id_club_away,home_score,away_score</p>
  
  <p>now I have my scraped data in a pandas dataframe, but with the following fields:</p>
  
  <p>gamedate,country,home_club,away_club,home_score,away_score</p>
  
  <p>So I want to insert this data in the 3 tables where it checks if the country and clubs already exist in the database, if not add them to the appropriate tables and then replace all text fields with the ids and insert the match data.</p>
  
  <p>Is this best done by loading the country and club tables in python and transforming the data with pandas/python or is there a better/more efficient way. I hope to build this quite scalable to be able to quickly load large datasets to learn for future projects.</p>