title: How to read parquet results from S3 which are paginated
body: <p>My results are stored in Amazon S3 in parquet format. </p>
  
  <p>My Requirements are as follows :</p>
  
  <ol>
  <li>I have a S3 bucket where I store my result as parquet (multiple
  parquet parts). I want to retrieve the results in all the parts.</li>
  <li>I want to retrieve all rows (in all the parts) as they are. (Doing query would be nice)</li>
  <li>My desire to paginate comes from my environment which is non distributed. I have an EC2 instance that has java code to get the results. I need the results to be paginated so that the EC2 host does not crash while retrieving the result.</li>
  </ol>
  
  <p>Options I looked into:</p>
  
  <ol>
  <li><p>ListObjectsV2Request - can't use this yet because we have not upgraded to AWS Java SDK 2.0</p></li>
  <li><p>Looking into S3 Select - Since S3 select needs the exact key of the contents I want to retrieve, first I will have to list all the parts from S3 and then use S3 Select on each part to get the results. Also I am not sure how I will paginate the input stream provided by S3</p></li>
  <li><p>Also looking into <a href="https://stackoverflow.com/questions/46831273/read-parquet-data-from-aws-s3-bucket">Read parquet data from AWS s3 bucket</a> but I am not clear on how to paginate the results.</p></li>
  </ol>
  
  <p>Any input/help will be highly appreciated.</p>