title: Python: how to mock a kafka topic for unit tests?
body: <p>We have a message scheduler that generates a hash-key from the message attributes before placing it on a Kafka topic queue with the key.</p>
  
  <p>This is done for de-duplication purposes. However, I am not sure how I could possibly test this deduplication without actually setting up a local cluster and checking that it is performing as expected.</p>
  
  <p>Searching online for tools for mocking a Kafka topic queue has not helped, and I am concerned that I am perhaps thinking about this the wrong way.</p>
  
  <p>Ultimately, whatever is used to mock the Kafka queue, should behave the same way as a local cluster - i.e. provide de-deuplication with Key inserts to a topic queue.</p>
  
  <p>Are there any such tools?</p>