title: Read large csv file from S3 into R
body: <p>I need to load a 3 GB csv file with about 18 million rows and 7 columns from S3 into R or RStudio respectively. My code for reading data from S3 usually works like this:</p>
  
  <pre><code>library("aws.s3")
  obj &lt;-get_object("s3://myBucketName/aFolder/fileName.csv")  
  csvcharobj &lt;- rawToChar(obj)  
  con &lt;- textConnection(csvcharobj)  
  data &lt;- read.csv(file = con)
  </code></pre>
  
  <p>Now, with the file being much bigger than usual, I receive an error</p>
  
  <pre><code>&gt; csvcharobj &lt;- rawToChar(obj)  
  Error in rawToChar(obj) : long vectors not supported yet: raw.c:68
  </code></pre>
  
  <p>Reading <a href="https://stackoverflow.com/questions/24335692/large-matrices-in-r-long-vectors-not-supported-yet">this post</a>, I understand that the vector is too long but how would I subset the data in this case? Any other suggestion how to deal with larger files to read from S3?</p>