none of us who had the misfortune to be in his presence can fail to remember the terror , the fear of a disagreement with him which had already cost the lives of twenty good and brave men in duels and other private misfortunes .