Take the 2-minute tour ×

I just figured out how to send jobs to be processed on machines on the cluster by using Condor. Since we have a lot of machines and not each of those machines are configured the same, I was wondering:

Is it possible to tell condor only to dispatch my jobs (python scripts) to machines, that have numpy installed on them since my script depends on this package?

share|improve this question

2 Answers 2

Like any other machine attribute, you just need to advertise it in the machine classad, and then have your jobs require it.

To advertise it in the machine classad, you can either hard-code it into each machine's condor config file by adding something like this:

has_numpy = True

... or better yet, you can tell Condor to dynamically discover it at runtime with a script and advertise the result via a startd classad hook. To do that, install a simple has_numpy script on each machine like so:

#!/usr/bin/env python
   import numpy
except ImportError:
   print "has_numpy = False"
   print "has_numpy = True"

... and then tell Condor to run it every five minutes and stick the results in the startd classad, by adding the following to the machine's condor config file:

HASNUMPY = /usr/libexec/condor/has_numpy

...and then ta-da (after a reconfig) your machines will dynamically detect and report whether numpy is installed and available to python scripts.

Then you just need to add a corresponding requirement to your job submit files, like so:

Requirements = (has_numpy == True)

...and your job will only run on machines where numpy is installed.

share|improve this answer

Do you need to? According to the condor manual:

Condor does not require an account (login) on machines where it runs a job. Condor can do this because of its remote system call technology, which traps library calls for such operations as reading or writing from disk files. The calls are transmitted over the network to be performed on the machine where the job was submitted.

To me this implies that if the machine submitting the job has numpy installed, it should work.

share|improve this answer
Thanks for your reply. Unfortunatly my case proves other wise. The jobs are submitted to different machines. Some submissions result in an ImportError of numpy. I double checked those machines and they had no numpy on them installed. So condor does not seem to prevent submitting a job to a machine which does not fulfil the requirements of the job - in my case a numpy installation. Maybe it is the case in our installation of condor, though. I myself didn't set up the system and it is the first time I work with condor. :-) –  Aufwind Mar 26 '12 at 22:01
Time to contact the condor developers. Either their manual is not correct, or Python is treated differently like Java is. –  Roland Smith Mar 26 '12 at 22:10
This quote from the manual is out of context. The Condor remote system call feature is available only to "standard universe" jobs. Although he doesn't say so, the OP must use the "vanilla" universe because the standard universe imposes constraints that preclude interpreters like Python from running under it. –  Pinko Mar 30 '12 at 2:15

Your Answer


