title: Repeating same processes for multiple csv files
body: <p>I am a beginner of Python. I have about 1000 CSV files (1.csv, 2.csv....1000.csv). Each CSV file has about 3,000,000,000 rows and 14 variables. I would like to clean data in each CSV file first using the same process for each CSV file: </p>
  
  <ol>
  <li>sum variable A and variable B,</li>
  <li>count C by sorting date, if the number of records in C for one day is greater than 50, then drop it. </li>
  </ol>
  
  <p>Next, save the cleaned data into a new CSV file. At last, append all 1000 new CSV files into one CSV file. </p>
  
  <p>I have some code as follows, but it imports all CSV files first, then process to clean data, which is very inefficient. I would like to clean the data in each CSV file first, then append new CSV files. Can anyone help me on this? Any help will be appreciated.  </p>
  
  <p><a href="https://i.stack.imgur.com/mNxL7.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/mNxL7.png" alt="enter image description here"></a></p>