Southworth was born in Harvard, Nebraska, to Orlando and Mariah Southworth and raised in Columbus, Ohio. He had four older brothers who played baseball. Before he was old enough to play with them, Southworth would give his old socks to his brothers so they could create makeshift balls. Southworth decided to play baseball against his father's wishes. Orlando Southworth had wanted his son to attend college. At the age of 19, he signed a contract with the Portsmouth team in the Ohio State League. He joined the Cleveland Indians in 1913, but only appeared in one game, entering as a replacement on defense.