title: Writing a system to monitor all nodes in a cluster
body: <p>This is a semi-experimental thing for me.</p>
  
  <p>I have a  cluster of over 100 (variable) nodes, and I want to write a monitoring application that would poll all the web nodes every n (eg 1 or 2) seconds, and record their response times.</p>
  
  <p>If the web node is already struggling, I may not want to bring it down by adding more requests. So it would be better if the poller has some intelligence to it.</p>
  
  <p>What language would you choose for such a project? Any open source projects that already do this that I can poke around? Any technical challenges that you can think of?</p>
  
  <p>I am starting to look at Hyperic HQ code, but man, that thing is huge.</p>