title: How to profile large datasets with Pandas profiling?
body: <p>Data is not perfectly clean, but is used without issue with pandas.
  The pandas library provides many extremely useful functions for EDA. </p>
  
  <p>But when I use profiling for large data i.e 100 million records with 10 columns, reading it from a database table, it does not complete and my laptop runs out of memory, the size of data in csv is around 6 gb and my RAM is 14 GB my idle usage is around 3 - 4 GB approximately.</p>
  
  <pre><code>df = pd.read_sql_query("select * from table", conn_params)
  profile = pandas.profiling.ProfileReport(df)
  profile.to_file(outputfile="myoutput.html")
  </code></pre>
  
  <p>I have also tried with <code>check_recoded = False</code> option as well.
  But it does not help in profiling entirely.
  Is there any way to chunk and read the data and finally generate the summary report as a whole? OR any other method to use this function with large dataset.</p>