title: Throttling in a queue-based distribution system to manage SLA
body: <p>I have a system is composed of a set of services which communicate which tasks to work on using SQS queues. We generate an artifact for a subset of customers (Group A) now, but want to start generating for all other customers as well (Group B), a ten-fold increase. The flow goes like this: 1. New inputs available from upstream dependency, we receive message in our work queue 2. Grab task from queue, do some processing, place outputs in another service's input queue 3. Repeat</p>
  
  <p>We don't have the capacity to generate a new artifact every time we receive new input, so I'm thinking of solving this by decoupling the input rate from our consumption rate. We'd like to still continue generating artifacts for Group A at the same cadence they currently receive them (every 6 hours). It's sufficient to refresh the artifacts for Group B once a day. If we can control our own refresh frequency, we can consume inputs at a rate that we know we can support.</p>
  
  <p>Since our queue-based systems do not keep track of any sort of state, I am thinking of making them more stateful - record when the last time we attempted to generate an artifact was, and don't generate a new version if it hasn't been 6 hours (Group A) or 24 hours (Group B). The problem is different accounts have drastically different processing times (ranging from minutes to over 12 hours) and because these calls are asynchronous, we currently have no way of knowing if/when an artifact was generated - it is best-effort, and if a particular run fails we wait for the next set of inputs and try generating using the new inputs.</p>
  
  <p>I will be, in effect, implementing my own throttling and deciding when to process a message vs dropping it. Throttling the inputs to the system is not an option, because dropping requests before examining their contents could cause us to miss a message and miss our SLA.</p>
  
  <p>Is this a good approach to thinking about the problem?</p>