Harrison's first wife was Carrie Harrison. After she died, he married Mary Dimmick Harrison. During the American Civil War Harrison was a colonel in the Union Army (later a brevet brigadier general). He was also a successful lawyer, arguing many cases before the United States Supreme Court.