Smith was born in Salt Lake City on July 18, 1876, as the first son of Julina Lambson Smith, the second wife and first plural wife of Joseph F. Smith, then a member of the Quorum of the Twelve. By agreement between his parents, Smith was given his father's name, even though Joseph F. Smith's third and fourth wives had previously had sons. Growing up, Smith lived in his father's large family home at 333 West 100 North in Salt Lake City, Utah Territory. The house was opposite the original campus of the University of Deseret (modern University of Utah), on a site now occupied by the LDS Business College. He also often worked on the family farm in Taylorsville, Utah, as a child.