title: Spark streaming if(!rdd.partitions.isEmpty) not working
body: <p>I'm trying to create a dStream from a kafka server and then do some transformations on that stream. I have included a catch for if the stream is empty (<code>if(!rdd.partitions.isEmpty)</code>); however, even when no events are being published to the kafka topic, the <code>else</code> statement is never reached.</p>
  
  <pre><code>val stream = KafkaUtils.createDirectStream[String, String, StringDecoder, StringDecoder](ssc, kafkaParams, topics)
  
  stream.foreachRDD { rdd =&gt;
      if(!rdd.partitions.isEmpty) {
  
          val message = rdd.map((x$2) =&gt; x$2._2).collect().toList.map(parser)
  
          val val = message(0)
  
      } else println("empty stream...")
  
      ssc.start() 
      ssc.awaitTermination()
  
  }
  </code></pre>
  
  <p>Is there an alternative statement I should use to check if the stream is empty when using <code>KafkaUtils.createDirectStream</code> rather than <code>createStream</code>?</p>