Graham spent the next eighteen years as a quiet country gentleman, spending his time on riding and sports, studying the classics and making occasional visits to London and Edinburgh. When his carriage was stopped in Park Lane, London by a highwayman demanding money, jewels, and watches at gunpoint, while two accomplices seized the horses' heads, Graham, who was on the opposite side of the carriage, leapt across the ladies to the carriage-door, and collaring the assailant, threw him to the ground. Then, drawing his sword, which at the time formed part of a dress suit, he threatened to run the man through, if his associates holding the horses' heads attempted to come to his assistance. They immediately fled, and the prostrate highwayman was arrested.