title: Triggering an AWS Lambda Function based on multiple (1k-10k) schedules
body: <p>We have an AWS Lambda function which queries some data for a client from our DB and sends a report to the client. Some clients want daily reports, some might need weekly or monthly reports. The number of clients can go up ~1000 and each client might have ~10 such reports.</p>
  <p>So we are looking for a way to trigger the Lambda function with different parameters based on schedules set by each client.</p>
  <p><strong>For Example</strong>:</p>
  <p>Client A wants daily report of their data to be sent to abc@clienta.com and Client B wants a weekly report of their data to be sent to xyz@clientb.com. So the Lambda function will be invoked twice on Sunday 12 AM (for both clients) and once on Monday-Saturday 12 AM (for Client A).</p>
  <p>We found the following solutions on AWS, but both have some limitations.</p>
  <p><strong>Approach 1: Use CloudWatch Events</strong></p>
  <p>We can create a CloudWatch Events Rule for each client and each report that could trigger our Lambda function on each schedule.</p>
  <p><strong>Pros</strong>:
  Simple setup, easy to implement.</p>
  <p><strong>Cons</strong>:</p>
  <p>There is a limitation of 100 Event Rules per AWS Account. It's mentioned that we can contact AWS to get it increased, but we are not sure if it can be increased to the number we are looking for (Currently it is ~10k, but we would prefer a solution in which there is no such limit). Also, a limit of 100 per account gives an indication that this is not a suitable solution for such a use case.</p>
  <p><strong>Approach 2: Using Step Functions</strong></p>
  <p>For each client and each report, we can create one AWS State Machine. We can use the Iterator pattern in Step Functions to wait for a day/week/month and then re-invoke the Lambda Function.</p>
  <p><strong>Pros</strong>:
  No limitations on number of State Machines, so this enables us to scale easily.</p>
  <p><strong>Cons</strong>:</p>
  <p>Step Functions have a limitation that they can run for a year, at maximum. This will be a problem in our case because the users will need to get the reports for a much longer period. There is a way to overcome this in Step Functions. Just before it's about to reach the 1-year limit, we can cancel the execution and start a fresh execution. So overall, this solution looks complex.</p>
  <p>Can someone suggest a better solution for this on AWS?</p>