title: Spark Streaming - Best way to Split Input Stream based on filter Param
body: <p>I currently try to create some kind of monitoring solution - some data is written to kafka and I read this data with Spark Streaming and process it.</p>
  
  <p>For preprocessing the data for machine learning and anomaly detection I would like to split the stream based on some filter Parameters. So far I have learned that DStreams themselves cannot be split into several streams.</p>
  
  <p>The problem I am mainly facing is that many algorithms(like KMeans) only take continues data and not discrete data like e.g. url or some other String.  </p>
  
  <p>My requirements would ideally be: </p>
  
  <ul>
  <li>Read data from kafka and generate a list of strings based on what I read</li>
  <li>Generate multiple streams based on that list of strings - (split stream, filter stream, or whatever is best practice)</li>
  <li>Use those streams to train different models for each Stream to get a baseline and afterwards compare everything that comes later against the baseline</li>
  </ul>
  
  <p>I would be happy to get any suggestions how to approach my problem. I cannot imagine that this scenario is not covered in Spark - however until now I did not discover a working solution.</p>