title: How to make prediction with sagemaker on pandas dataframe
body: <p>I am using Sagemaker to train and deploy my machine learning model. As regard to prediction, it will be executed by a lambda function as a scheduled job (every hour). The process is as follows:</p>
  
  <ol>
  <li>pull new data from S3 since last prediction </li>
  <li>preprocess, aggregate and create prediction data set </li>
  <li>call sagemaker endpoint and make prediction </li>
  <li>either save result to s3 or insert to database table</li>
  </ol>
  
  <p>Based on my finding, typically the input will either from lambda payload </p>
  
  <pre><code>data = json.loads(json.dumps(event))
  payload = data['data']
  print(payload)
  
  response = runtime.invoke_endpoint(EndpointName=ENDPOINT_NAME,
                                     ContentType='text/csv',
                                     Body=payload)
  </code></pre>
  
  <p>or read from s3 file:
  my_bucket = resource.Bucket('pred_data') #subsitute this for your s3 bucket name. </p>
  
  <pre><code>obj = client.get_object(Bucket=my_bucket, Key='foo.csv')
  lines= obj['Body'].read().decode('utf-8').splitlines()
  reader = csv.reader(lines)
  file = io.StringIO(lines)
  
  
  response = runtime.invoke_endpoint(EndpointName=ENDPOINT,
                                     ContentType='*/*',
                                     Body = file.getvalue(),
                                     Body=payload)
  output = response['Body'].read().decode('utf-8')
  </code></pre>
  
  <p>Since I will be pulling raw data from s3 and preprocess, a <code>pandas</code> dataframe will be generated. Is it possible to feed this directly as the input of <code>invoke_endpoint</code>? I could upload the aggregated dataset to another S3 bucket, but does it have to go through the <code>decoding</code>, <code>csv.reader</code>, <code>StringIO</code> and all that just like the example I found or is there an easy way to do it? Is the <code>decode</code> step really necessary to get the output?</p>