title: Event Sourcing with Kinesis - Replaying and Persistence
body: <p>I am trying to implement an event-driven architecture using Amazon Kinesis as the central event log of the platform. The idea is pretty much the same to the one presented by <a href="https://read.acloud.guru/serverless-event-sourcing-at-nordstrom-ea69bd8fb7cc" rel="noreferrer">Nordstrom's with the Hello-Retail project</a>.</p>
  
  <p>I have done similar things with Apache Kafka before, but Kinesis seems to be a cost-effective alternative to Kafka and I decided to give it a shot. I am, however, facing some challenges related to event persistence and replaying. I have two questions: </p>
  
  <ol>
  <li>Are you guys using Kinesis for such use-case OR do you recommend using it?</li>
  <li>Since Kinesis is not able to retain the events forever (like Kafka does), how to handle replays from consumers?</li>
  </ol>
  
  <p>I'm currently using a lambda function (Firehose is also an option) to persist all events to Amazon S3. Then, one could read past events from the storage and then start listening to new events coming from the stream. But I'm not happy with this solution. Consumers are not be able to use Kinesis' checkpoints (Kafka's consumer offsets). Plus, Java's <a href="https://github.com/awslabs/amazon-kinesis-client/issues/133" rel="noreferrer">KCL does not support the AFTER_SEQUENCE_NUMBER yet</a>, which would be useful in such implementation.</p>