title: Performing a POST REST request using Pyspark
body: <p>I have a pyspark script as part of an oozie job. The actions are as follows:
  1. Fetch data from a data store.
  2. Some data munging operations on the cluster.
  3. Download the data. Currently, doing a toPandas() on the pyspark dataframe.[Doing this to easily convert to a json dump]
  4. Pushing the data to a REST API.</p>
  
  <p>Step (3) in only necessitated because of step(4) as I you need the data to be on the driver to be able to do a REST call. However, I have noticed that step(3) is responsible for the variable execution time of my script as well slowing down of my script. My question is it possible to invoke and POST to a REST API from the worker nodes? I saw some examples of using a GET request from REST(<a href="https://dataplatform.cloud.ibm.com/analytics/notebooks/52845a4a-1b5e-4f6e-b1a3-f312d796a93a/view?access_token=e3f303d7dd90138a9cf1fb77b00265a7b02aa12b891c2018e2e547f2050ef4e0" rel="nofollow noreferrer">https://dataplatform.cloud.ibm.com/analytics/notebooks/52845a4a-1b5e-4f6e-b1a3-f312d796a93a/view?access_token=e3f303d7dd90138a9cf1fb77b00265a7b02aa12b891c2018e2e547f2050ef4e0</a>), but this did not work for my usecase.</p>