Take the 2-minute tour ×

I'd like to be able to create a "job" that will execute in an arbitrary time from now... Let's say 1 year from now. I'm trying to come up with a stable, distributed system that doesn't rely on me maintaining a server and scheduling code. (Obviously, I'll have to maintain the servers to execute the job).

I realize I can poll simpleDB every few seconds and check to see if there's anything that needs to be executed, but this seems very inefficient. Ideally I could create an Amazon SNS topic that would fire off at the appropriate time, but I don't think it's possible.

Alternatively, I could create a message in the Amazon SQS that would not be visible for 1 year. After 1 year, it becomes visible and my polling code picks up on it and executes it.

It would seem this is a topic like Singletons or Inversion Control that Phd's have discussed and come up with best practices for. I can't find the articles if there any.

Any ideas?


share|improve this question
add comment

2 Answers

up vote 0 down vote accepted

I think it really depends on what kind of job you want to execute in 1 year and if that value (1 year) is actually hypothetical. There are many ways to schedule a task, windows and linux both offer a service to schedule tasks. Windows being Task Scheduler, linux being crontab. In addition to those operating system specific solutions you can use Maintenance tasks on MSSQL server and I'm sure many of the larger db's have similar features.

Without knowing more about what you plan on doing its kind of hard to suggest any more alternatives since I think many of the other solutions would be specific to the technologies and platforms you plan on using. If you want to provide some more insight on what you're going to be doing with these tasks then I'd be more than happy to expand my answer to be more helpful.

share|improve this answer
Hi & thanks @bwight- A hypothetical job might be sending an email one year from now. I want to create it now, and send it one year later. Ideally, I could offload as much of the infrastructure to AWS as possible, so I don't have to run and maintain a task scheduler. I'm getting the sense it's not possible purely off Amazon Web Services without maintaining my own EC2. –  Hairgami_Master Feb 27 '12 at 18:28
Something like sending an email could be achieved using simpledb as a storage device and then polling it to see if anything needs to be updated. Being that it seems your tasks are set for dates in the very distant future it wouldn't be necessary to check the database every few seconds. Setting a scheduled task or crontab to run once a day and send any emails would work just fine. It would not be possible without ec2 instances. If you didn't want to deal with setting up the distributed fault tolerant environment yourself you can look into AWS Elastic Beanstalk if you know Java. –  bwight Feb 27 '12 at 19:36
Awesome thanks @bwight. I'm going to need more timing accuracy, so I think what I'll do is check once a day to see if there are any jobs that need to run that day. If so, I'll create a local scheduler for that job using Quartz scheduler. Cheers! –  Hairgami_Master Feb 27 '12 at 20:06
add comment

The easiest way for most people to do this would be to run at least an EC2 server with a cron job on the EC2 server to trigger an action. However, the cost of running an EC2 server 24 hours a day for a year just to trigger an action would be around $170 at the cheapest (8G t1.micro with Heavy Utilization Reserved Instance). Plus, you have to monitor that server and recover from failures.

I have sketched out a different approach to running jobs on a schedule that uses AWS resources completely. It's a bit more work, but does not have the expense or maintenance issues with running an EC2 instance.

You can set up an Auto Scaling schedule (cron format) to start an instance at some point in the future, or on a recurring schedule (e.g., nightly). When you set this up, you specify the job to be run in a user-data script for the launch configuration.

I've written out sample commands in the following article, along with special settings you need to take care of for this to work with Auto Scaling:

Running EC2 Instances on a Recurring Schedule with Auto Scaling

With this approach, you only pay for the EC2 instance hours when the job is actually running and the server can shut itself down afterwards.

This wouldn't be a reasonable way to schedule tens of thousands of emails with an individual timer for each, but it can make a lot of sense for large, infrequent jobs (a few times a day to once per year).

share|improve this answer
Thanks Eric- I like your style. That is an insane solution to an insane problem! Cheers! –  Hairgami_Master Feb 28 '12 at 4:51
Great Thank you –  Vor Apr 4 '13 at 21:26
add comment

Your Answer


