title: Proper way to programmatically stop an Alpakka Kafka stream
body: <p>We are trying to use Akka Streams with Alpakka Kafka to consume a stream of events in a service. For handling event processing errors we are using Kafka autocommit and more than one queue. For example, if we have the topic <code>user_created</code>, which we want to consume from a products service, we also create <code>user_created_for_products_failed</code> and <code>user_created_for_products_dead_letter</code>. These two extra topics are coupled to a specific Kafka consumer group. If an event fails to be processed, it goes to the failed queue, where we try to consume again in five minutes--if it fails again it goes to dead letters.</p>
  
  <p>On deployment we want to ensure that we don't lose events. So we are trying to stop the stream before stopping the application. As I said, we are using autocommit, but all of these events that are "flying" are not processed yet. Once the stream and application are stopped, we can deploy the new code and start the application again.</p>
  
  <p>After reading the documentation, we have seen the <a href="https://doc.akka.io/docs/akka/2.5/stream/stream-dynamic.html#controlling-stream-completion-with-killswitch" rel="nofollow noreferrer"><code>KillSwitch</code></a> feature. The problem that we are seeing in it is that the <code>shutdown</code> method returns <code>Unit</code> instead <code>Future[Unit]</code> as we expect. We are not sure that we won't lose events using it, because in tests it looks like it goes too fast to be working properly.</p>
  
  <p>As a workaround, we create an <code>ActorSystem</code> for each stream and use the <code>terminate</code> method (which returns a <code>Future[Terminate]</code>). The problem with this solution is that we don't think that creating an <code>ActorSystem</code> per stream will scale well, and <code>terminate</code> takes a lot of time to resolve (in tests it takes up to one minute to shut down).</p>
  
  <p>Have you faced a problem like this? Is there a faster way (compared to <code>ActorSystem.terminate</code>) to stop a stream and ensure that all the events that the <code>Source</code> has emitted have been processed?</p>