title: How to stream large files through Kafka?
body: <p>I'm in the process of migrating an ACID-based monolith to an event-based microservice architecture. In the monolith potentially large files are stored in a database and I want to share this information (including the file content) with the microservices.</p>
  
  <p>My approach would be to split the file into numbered blocks and send several messages (e.g. 1 <code>FileCreatedMessage</code> with metadata and an id followed by <code>n</code> <code>FileContentMessage</code> containing the block and its sequence number). On the receiving side messages may not arrive in order. Therefore I'd store the blocks from messages, order and join them and store the result.</p>
  
  <p>Is there any approach which allows me to stream the data through Kafka with one message or another approach without the overhead of implementing the spliting, order and join logic for several messages?</p>
  
  <p>I noticed Kafka Streams. It seems to solve different problems than this one.</p>