<p>We are writing a package delivery tracker application using Kubernetes/go-lang. Everything's fine until we've reached a road block where we have certain scheduling needs, and couldn't figure out what to use. The use case is:</p>

<ol>
<li>Once a package is assigned and out for delivery, its location should be tracked.</li>
<li>If the package is not assigned after 3 hours of reaching the warehouse, we should mark it as "delivering tomorrow".</li>
<li>Some usual cron-type jobs.</li>
</ol>

<p>for #1 we are planning to schedule a job which will poll the location of package after every 10 seconds. But this is a one time job. Once the package has been delivered. It should die out.</p>

<p>for #2 we are planning to schedule a timed/cron job which will be triggered by our API. This job will be scheduled for 3 hours after the receipt of package at warehouse.</p>

<p>Our #3 above is a usual cron-job for various internal purposes.</p>

<p>We want to use a single scheduling platform to serve all these type of requirements.</p>

<p>Since we are using Kubernetes exclusively, we want to leverage it's job scheduling functionality. But we have certain doubt about it.</p>

<ol>
<li>Can we create these jobs from our source code ? It is possible as mentioned <a href="https://github.com/samuell/k8s-experiments/blob/0ada4a53c9e19d29f129960b5d278afe99cdb67f/k8sexp.go#L43-L90" rel="nofollow noreferrer" title="here">here</a>. But I am not sure whether our system administrator would allow us to do that.</li>
<li><p>I've read about work queues in Kubernetes, where we can push our jobs to a work queue and a consumer will create jobs for these work items. </p>

<p>But, I am unaware of how to create a permanent consumer daemon for that work queue, which will poll that queue and create jobs for each work item.</p>

<p>Another doubt is how to schedule simple cron jobs(#3 above) here.</p></li>
</ol>

<p>I've also heard about <a href="https://github.com/ajvb/kala" rel="nofollow noreferrer">Kala</a>, but not sure how will this fit in Kubernetes world.</p>

<p>Any reference, pointers, links or suggestions would be highly appreciated as I am pretty new to Kubernetes/Golang in general, and not finding anything concrete on google as well.</p>

diwakarb
<p>How many packages are there?  How important are the time intervals you mention?  There are several approaches that come to mind:</p>

<p><strong>Kubernetes <a href="https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/" rel="nofollow noreferrer">cron jobs</a></strong> can run on a schedule, and presumably solve your third need.  If you have "some" packages (I wouldn't necessarily try this with more than "hundreds") then for each package you could create a new cron job which runs once then deletes itself.  Cron jobs can't run more often than once a minute.  If you can relax your "poll once every 10 seconds" requirement to "poll once a minute" and the number of packages is small, you could do everything this way.</p>

<p>For the second requirement, you can <strong>check every package for expiration on an interval</strong>.  This works better if the "after 3 hours" requirement isn't particularly strict.  Write a program that looks at <em>every</em> package, and if the three-hour delay has passed, changes its state, then write a cron job that runs this every 15 minutes.  The scaling issues around this are different, but "find expired packages" sounds like a single SQL query; it depends on your backend technology.</p>

<p>You can write <strong>one long-running program that does all of the interval-based checks</strong>.  You hint at using Go, and you can set up a <code>time.Timer</code> that triggers every 10 seconds (either one for the entire program, or one per package).  I would probably have the program know, in memory, each package ID, its current state, and when it needs to change state to "delivering tomorrow", and run all of this in a single process.  An ordinary Deployment would manage this.  You should also keep this state in a database, and that might introduce some scaling limits.</p>

<p>Finally, you can <strong>use a message queueing system</strong>.  <a href="http://www.rabbitmq.com" rel="nofollow noreferrer">RabbitMQ</a> is a popular open-source choice for this.  You can arrange for messages to be delayed (send a message to exchange <code>package.in</code> that routes to a queue <code>package.wait</code> with a <a href="http://www.rabbitmq.com/ttl.html" rel="nofollow noreferrer">TTL</a> of 10 seconds, set that queue's <a href="http://www.rabbitmq.com/dlx.html" rel="nofollow noreferrer">dead-letter exchange</a> to <code>package.ready</code>, send that to a queue <code>package.out</code>, and have applications consume from that queue).  Again you'd write a long-running process; this one gets items from the queue, retrieves their state from the database, polls for their current location, and marks them as "delivered tomorrow" or requeues them in <code>package.in</code> as appropriate.  This is the most scalable answer (you can launch many workers via a Kubernetes Deployment) but also makes the least use of inbuilt Kubernetes parts.</p>
