title: Spark fastest way for creating RDD of numpy arrays
body: <p>My spark application is using RDD's of numpy arrays.<br>
  At the moment, I'm reading my data from AWS S3, and its represented as 
  a simple text file where each line is a vector and each element is seperated by space, for example:</p>
  
  <pre><code>1 2 3
  5.1 3.6 2.1
  3 0.24 1.333
  </code></pre>
  
  <p>I'm using numpy's function <code>loadtxt()</code> in order to create a numpy array from it.
  <br>However, this method seems to be very slow and my app is spending too much time(I think) for converting my dataset to a numpy array.</p>
  
  <p>Can you suggest me a better way for doing it? For example, should I keep my dataset as a binary file?,
  should I create the RDD in another way?</p>
  
  <p>Some code for how I create my RDD:</p>
  
  <pre><code>data = sc.textFile("s3_url", initial_num_of_partitions).mapPartitions(readData)
  </code></pre>
  
  <p>readData function:</p>
  
  <pre><code> def readPointBatch(iterator):
       return [(np.loadtxt(iterator,dtype=np.float64)]
  </code></pre>