Breckinridge's family split on the issue, with two of his sons – Joseph and Charles – fighting for the Union cause, and two – Willie and Robert Jr. – fighting for the Confederacy. While three of his sons-in-law also fought for the Union, daughter Sophonisba's husband, Theophilus Steele, rode with John Hunt Morgan, and it is likely that Robert Breckinridge's intervention kept him from being executed by Edwin M. Stanton. Following the war, Willie Breckinridge's wife Issa refused to let her father-in-law see two of his grandchildren for a period of two years.