title: Boto3 list_object_v2 painfully slow
body: <p>I am trying to use the Boto3 API (specifically, list_objects_v2, as this is what's recommended) to get all keys on my buckets. I executed 8 tasks in parallel on each bucket that I have on S3 totaling about ~55Gb of data. I have been running this for 16+ hours and none of my calls have returned. Is this expected behavior? If it even has to download the entire 55Gb it should not take more than a few hours to download at most (I'm on a very fast internet connection). </p>
  
  <p>Is AWS rate limiting me in some abnormal way? Their documentation says there is a limit of 5,500 requests per second. Since I'm looking at the order of ~5,000,000 S3 items, if I am being rate limited by my estimate the rate limiting given an infinitely fast connection shouldn't lower bound my transfer time at more than 15 minutes. So this is not the issue?</p>
  
  <p>As an aside, this operation appears to be monopolizing my CPU. The below code is what I am executing for this call. Is there something blatantly obvious I am missing here? Looks to me that it's spending all of it's time in the call to <code>list_objects_v2</code>. I'm no AWS guru so it's possible what I'm doing is super bad for <em>reasons</em> I don't know.</p>
  
  <pre><code>def list_all_keys(self):
          reached_end = False
          all_keys = []
          token = None
          while not reached_end:
              # will limit to 1000 objects
              response = self.client.list_objects_v2(Bucket=self.bucket)
              token = response.get('NextContinuationToken', None)
              contents = response.get('Contents', [])
              all_keys.extend([obj['Key'] for obj in contents])
              if not token and response.get('IsTruncated') is False:
                  reached_end = True
          return all_keys
  </code></pre>