title: TCP Socket Read Variable Length Data w/o Framing or Size Indicators
body: <p>I am currently writing code to transfer data to a remote vendor. The transfer will take place over a TCP socket. The problem I have is the data is variable length and there are no framing or size markers. Sending the data is no problem, but I am unsure of the best way to handle the returned data.</p>
  
  <p>The data is comprised of distinct "messages" but they do not have a fixed size. Each message has an 8 or 16 byte bitmap that indicates what components are included in this message. Some components are fixed length and some are variable. Each variable length component has a size prefix for that portion of the overall message.</p>
  
  <p>When I first open the socket I will send over messages and each one should receive a response. When I begin reading data I should be at the start of a message. I will need to interpret the bitmap to know what message fields are included. As the data arrives I will have to validate that each field indicated by the bitmap is present and of the correct size. </p>
  
  <p>Once I have read all of the first message, the next one starts. My concern is if the transmission gets cut partway through a message, how can I recover and correctly find the next message start?</p>
  
  <p>I will have to simulate a connection failure and my code needs to automatically retry a set number of times before canceling that message.</p>
  
  <p>I have no control over the code on the remote end and cannot get framing bytes or size prefixes added to the messages.</p>
  
  <p>Best practices, design patterns, or ideas on the best way to handle this are all welcomed.</p>