Late in life, Southworth served as a scout for the Braves. He endured a great deal of tragedy in his baseball career, first experiencing the stillbirth of his twin babies and the deaths of his wife and his adult son. He died in 1969. He was inducted into the Baseball Hall of Fame in 2008. Six years later, the Cardinals inducted him into the St. Louis Cardinals Hall of Fame Museum. He also belongs to the Alabama Sports Hall of Fame. Southworth had a son, Billy Southworth Jr., and a cousin, Bill Southworth, who both played professional baseball.