title: Spark - Sort DStream by Key and limit to 5 values
body: <p>I've started to learn <code>spark</code> and I wrote a <code>pyspark</code> streaming program to read stock data (symbol, volume) from port <code>3333</code>.</p>
  
  <p>Sample data streamed at <code>3333</code></p>
  
  <pre><code>"AAC",111113
  "ABT",7451020
  "ABBV",7325429
  "ADPT",318617
  "AET",1839122
  "ALR",372777
  "AGN",4170581
  "ABC",3001798
  "ANTM",1968246
  </code></pre>
  
  <p>I want to display the top 5 symbols based on <code>volume</code>. So I used a mapper to read each line, then split it by <code>comma</code> and reversed. </p>
  
  <pre><code>from pyspark import SparkContext
  from pyspark.streaming import StreamingContext
  
  sc = SparkContext("local[2]", "NetworkWordCount")
  ssc = StreamingContext(sc, 5)
  
  lines = ssc.socketTextStream("localhost", 3333)
  stocks = lines.map(lambda line: sorted(line.split(','), reverse=True))
  stocks.pprint()
  </code></pre>
  
  <p>Following is the output of <code>stocks.pprint()</code></p>
  
  <pre><code>[u'111113', u'"AAC"']
  [u'7451020', u'"ABT"']
  [u'7325429', u'"ABBV"']
  [u'318617', u'"ADPT"']
  [u'1839122', u'"AET"']
  [u'372777', u'"ALR"']
  [u'4170581', u'"AGN"']
  [u'3001798', u'"ABC"']
  [u'1968246', u'"ANTM"']
  </code></pre>
  
  <p>I've got the following function in mind to display the stock symbols but not sure how to sort the stocks by key(<code>volume</code>) and then limit the function to display only first 5 values. </p>
  
  <pre><code>stocks.foreachRDD(processStocks)
  
  def processStocks(stock):
      for st in stock.collect():
          print st[1]
  </code></pre>