Alphonso's wife Fanny Phelps Taft died a year after the family moved to the Mount Auburn residence, in June 1852. In December 1853 Alphonso remarried, choosing a schoolteacher from Massachusetts named Louise Torrey. Louise Taft would give birth to their second child, William Howard Taft, in the house on September 15, 1857, presumably in the first-floor nursery in the rear ell. (The first child had died at age fourteen months from whooping cough.) Alphonso had six children living in the house, two by Fanny (three others had died beforehand) and four by Louise.