Benjamin Singleton married and was the father of several children. Two of his children – Emily, born about 1840 in Tennessee, and Sarah, born about 1858 in Michigan – wrote letters to their father from East Nashville, Tennessee, during the mid-1880s. An undated note regarding Singleton's testimony before the U.S. Senate Committee in 1880 quoted Singleton as saying: "I have been a slave fled to Canada when my children were small and nineteen years after when I returned they were grown."