He married Martha E McCarty in 1846 and they had two children, William and John Lea. Smith later married Lavinia W. Henderson, daughter of Hon. Luke Lea, U. S. Representative for Tennessee, in 1855.
In 1846, he married Martha E. McCarty, who had two children: William and John Lea. Later, in 1855, Smith wedded Lavinia W. Henderson, daughter of Hon. Luke Lea, U.S. Representative for Tennessee.