title: Copy Redshift table from S3 csv file using Python?
body: <p>What is the recommended module and syntax to programatically copy from an S3 csv file to a Redshift table?  I've been trying with the psycopg2 module, but without success (see <a href="https://stackoverflow.com/questions/41388555/psycopg2-copy-expert-how-to-copy-in-a-gzipped-csv-file">psycopg2 copy_expert() - how to copy in a gzipped csv file?</a>).  I've tried cur.execute(), cur.copy_expert() and cur.copy_from() - all unsuccessfully.   My experience and comments I've read lead me to conclude that psycopg2, while sufficient for python-programming a postgres DB, will not work for Redshift tables for some reason.  So what is the workaround if I want a Python script to do this copy?</p>
  
  <p>Here is the COPY statement I want to run.  The source is a gzipped csv file with a pipe delimiter.  This works fine from a SQL interface like DBeaver, but I can't figure out how it would translate to Python:</p>
  
  <pre><code>'''COPY &lt;destination_table&gt; from 's3://bucket/my_source_file.csv.gz' CREDENTIALS &lt;my credentials&gt;  delimiter '|' IGNOREHEADER 1 ENCODING UTF8 IGNOREBLANK    LINES NULL AS 'NULL' EMPTYASNULL BLANKSASNULL gzip ACCEPTINVCHARS timeformat 'auto' dateformat 'auto' MAXERROR 100 compupdate on;'''
  </code></pre>