title: AWS SQS BackUp Solution Design
body: <p><strong>Problem Statement</strong></p>
  
  <p>Informal State</p>
  
  <p>We have some scenarios where the integration layer (a combination of AWS SNS/SQS components and etc.) is also responsible for the data distribution to target systems. Those are mostly async flows.  In this case, we send a confirmation to a caller that we have received the data and will take a responsibility for the data delivery.  Here, although the data is not originated from the integration layer we are still holding it and need to make sure that the data is not lost, for example, if the consumers are down or if messages, on-error, are sent to the DLQs and hence being automatically deleted after the retention period.</p>
  
  <p><strong>Solution Design</strong></p>
  
  <p>Currently my idea was to proceed with a back-up of the SQS/DLQ queues based upon CloudWatch configured alerts using ApproximateAgeOfOldestMessage metric with some applied threshold (something like the below):</p>
  
  <blockquote>
    <p>Msg Expiration Event if ApproximateAgeOfOldestMessage / Message retention > Threshold</p>
  </blockquote>
  
  <p>Now, more I go forward with this idea and more I doubt that this might be actually the right approach…</p>
  
  <p>In particular,  I would like to build something unobtrusive that can be "attached" to our SQS queues and dump the messages that are about to expire in some repository, like for example the AWS S3. Then have a procedure to recover the messages from S3 to the same original queue.</p>
  
  <p>The above procedure contains many challenges like: message identification and consumption (receive message is not design to "query" for specific messages), message dump in the repository with a reference to the source queue, etc. which would suggest to me that the above approach might be a complex over-kill. </p>
  
  <p>That being said, I'm aware of other "alternatives" (such as <a href="https://stackoverflow.com/questions/57756954/create-amazon-sqs-backup-automatically-in-another-queue)">this</a>) but I would appreciate if you could answer to the specific technical details described above, <strong>without trying to challenge the "need" instead.</strong></p>