The war resulted in the destruction of Taylor's home, including his much prized library, and his sugar cane property and facilities. He moved his family to New Orleans at the end of the war and lived there until his wife died in 1875. He was president of The Boston Club 1868–1873. After his wife's death, he moved with their three daughters to Winchester, Virginia. From there he traveled regularly to see friends and colleagues in Washington, D.C. and New York City.