title: How to detect unresponsive/frozen processes?
body: <p>I have several scripts that I use to do some web crawling. They are always running, and should never stop. However, after about a week, they systematically "freeze": there is no output anymore, no response to Ctrl+C or anything. The only way is to <code>kill</code> the process and restart it.</p>
  
  <p>I suspect that these issues come from the library I use for retrieving the data (<code>urllib2</code>), but the issue is very hard to reproduce.</p>
  
  <p>I am thus wondering how I could check the state of the process and kill/restart it automatically if it is frozen. I was thinking of creating a PID file, and update it regularly. Another script could then periodically check the last modification date of this PID file, and restart the process if it's too old. I could use something like <a href="http://mmonit.com/monit/" rel="nofollow">Monit</a> to do the monitoring.</p>
  
  <p>Is this how I should do it? Is there another best practice/common way for checking the responsiveness of a process?</p>