title: Fetching data from postgres database in batch (python)
body: <p>I have the following Postgres query where I am fetching data from table1 with rows ~25 million and would like to write the output of the below query into multiple files.</p>
  
  <pre><code>query = """ WITH sequence AS (
                  SELECT 
                          a,
                          b,
                          c
                  FROM table1 )                    
  
  select * from sequence;"""
  
  </code></pre>
  
  <p>Below is the python script to fetch the complete dataset. How can I modify the script to fetch it to multiple files (eg. each file has 10000 rows)</p>
  
  <pre><code>#IMPORT LIBRARIES ########################
  import psycopg2
  from pandas import DataFrame
  
  #CREATE DATABASE CONNECTION ########################
  connect_str = "dbname='x' user='x' host='x' " "password='x' port = x"
  conn = psycopg2.connect(connect_str)
  cur = conn.cursor()
  conn.autocommit = True
  
  cur.execute(query)
  df = DataFrame(cur.fetchall())
  
  </code></pre>
  
  <p>Thanks</p>