title: How to get all the non-Zero values from a Dataframe[SparseVector] in Spark?
body: <p>I have a DataFrame[SparseVector] obtained from HashingTF from ml library in Spark. Below is my schema:</p>
  
  <pre><code>root
   |-- docId: string (nullable = true)
   |-- docNGrams: array (nullable = true)
   |    |-- element: string (containsNull = true)
   |-- HashedNGrams: vector (nullable = true)
  </code></pre>
  
  <p>HashedNGrams is a sparse Vector of 2^31 - 1 = 2147483647 number of features(The maximum). An example of HashedNGrams is:</p>
  
  <pre><code>[doc/00000.txt,(2147483647,[70921,235056,....],[1.0,2.0,...])
  [doc/00001.txt,(2147483647,[6067499,8758008,....],[1.0,1.0,...])
  ......
  ......
  </code></pre>
  
  <p>What I want is to get Just the Values:</p>
  
  <pre><code>70921,235056
  6067499,8758008
  .....  , ...
  </code></pre>
  
  <p>from this DataFrame to a Vector,List[Int], whatever, so to do some manipulation of the data. What I have tried so far is:
  1)Try to convert it to a dense Vector but it gives me an out of limits VM exception
  2)Tried every possible solution i found from here but with no lack at all! They all give me an error.
  3)Even save it in a file and then tried to take these values as a String(haven't completed this yet, but i feel it's Very wrong approach).</p>
  
  <p>Please help!</p>