text: Q:
  
  Automated test suite for linux servers and network devices?
  
  Coming from a development background i'm used to automated (unit) testing before stuff goes live.Now i'd like to use the same approach to (new) linux (and some windows) servers and networkdevices. 
  I want to be able to define tests / conditions which these systems should pass before they can go live, or run the tests against live hosts to check if they still comply with our standards.
  Tests I'd like to run go beyond networkscanning. For instance:
  
  I want to check if SSH is enabled, but no root login is allowed and keybased logins are enforced.
  On a printer I want to check if certain SNMP communities are set.
  On a linux host i'd want to check the ntp settings
  I'd want to be able to define custom check in some specific cases
  
  Do you know if such an automated system exist, which one would meet my requirements best? Or should I built on an existing unittest framework?
  
  A:
  
  This is what you use monitoring for.  There are plenty of monitoring systems, with different positives and negatives, and this isn't the place for an exhaustive discussion of the different options.  Your monitoring should, in principle, represent all of the assertions you make about your system, both in terms of outputs as well as response times.  I encourage the use of "monitoring-first systems administation" in my team, and the parallels to development should be obvious.
  Now, just as some things are hard to unit test, there are also some things that are hard to monitor.  Your SSH example is one of them -- while you can certainly try to login, and if that fails, then say you're done, but plenty of things could confound your test -- "try to login as root with a password fails" could be screwed up by someone changing the root password to test123 and turning on password auth -- you don't know what the password's been set to, so of course your test login will fail.
  For those things, you want a configuration management system, like Chef or Puppet.  These systems allow you to effectively make assertions about the state of a system (like "The PasswordAuthentication option in /etc/ssh/sshd_config should be set to false"), and the config management system will ensure that is the case each time it runs.  Good systems can also provide you with exception reporting ("hey, I thought you might like to know that PasswordAuthentication was set to true; it's OK, I fixed it, but you might want to go break someone's fingers") so you can know when something odd does happen.
  
  