the last thing arnold could want , as military governor of the city , was public scandal and disorder , the remaining loyalists in philadelphia threatened and frightened , the sons of liberty seen as secret vigilantes , a law unto themselves .