title: Sagemaker image classification: Best way to perform inference on many images in S3?
body: <p>I trained a model with the built-in RESnet18 docker image, and now I want to deploy the model to an endpoint and classify ~ 1 million images. I have all my training, validation, and test images stored on S3 in RecordIO format (converted with <a href="https://mxnet.incubator.apache.org/tutorials/basic/data.html?highlight=im2rec" rel="nofollow noreferrer">im2rec.py</a>). According to the <a href="http://docs.aws.amazon.com/sagemaker/latest/dg/image-classification.html" rel="nofollow noreferrer">docs</a>:</p>
  
  <blockquote>
    <p>The Amazon SageMaker Image Classification algorithm supports both RecordIO (application/x-recordio) and image (application/x-image) content types for training. The algorithm supports only application/x-image for inference.</p>
  </blockquote>
  
  <p>So I cannot perform inference on my training data in RecordIO format. To overcome this I copied all the raw .jpg images (~ 2GB) onto my Sagemaker Jupyter Notebook instance and performed inference one at a time in the following way:</p>
  
  <pre><code>img_list = os.listdir('temp_data') # list of all ~1,000,000 images
  
  for im in img_list:
      with open('temp_data/'+im, 'rb') as f:
          payload = f.read()
          payload = bytearray(payload)
      response = runtime.invoke_endpoint(EndpointName=endpoint_name, 
                                         ContentType='application/x-image', 
                                         Body=payload)
  
      etc...
  </code></pre>
  
  <p>Needless to say, transferring all the data onto my Notebook instance took a long time and I would prefer not having to do that before running inference. Why does the SageMaker Image Classification not support RecordIO for inference? And more importantly, what is the best way to run inference on many images without having to move them from S3?</p>