title: Amazon S3 - multipart upload vs split files-then-upload
body: <p>I am currently trying to upload files from local to S3 using python. I have extremely large files (over 10 GB) and when I went through some best practices for faster upload, I came across multipart upload. If I understood rightly, multipart upload does the below things:</p>
  
  <ol>
  <li>Split the file into a number of chunks.</li>
  <li>Upload each of these chunks to S3 (either serially or in parallel based on our code).</li>
  <li>Once the upload of each of these chunks are over, S3 takes care of the final assembling of individual chunks into a single final object/file.</li>
  </ol>
  
  <p>Since, after the uploads of all the chunks are over, it is obvious that multipart upload assembles everything into a single object. But, I want to keep the individual parts as it is or find another way to split the files and upload using python boto's put_object method. This is because, I want the individual chunks/parts of the file to be read in parallel from S3 for my further processing. Is there a way to do this or should I stick to the traditional way of splitting the file by myself and uploading them in parallel (for faster upload).</p>
  
  <p>Thanks in advance.</p>