Rathbone resigned from the Army in 1870, having risen to the rank of brevet colonel. After his resignation, he struggled to find and keep a job due to his mental instability. He became convinced that Harris was unfaithful. He also resented the attention Harris paid their children and reportedly threatened his wife on several occasions after suspecting that she was going to divorce him and take the children. Nonetheless, President Chester A. Arthur appointed Rathbone as U.S. Consul to the Province of Hanover in 1882. The family relocated to Germany, where Rathbone's mental health continued to decline.