Wheaton suffered after the death of his wife Ellen in 1858. His friends the Norths wrote from Minnesota in 1859, urging him to join them in Northfield, which they had founded. The following year, Wheaton moved there with many of his 12 children, joining other Syracuse families who had migrated to join the North family. When John North had earlier suffered financial failure in the Panic of 1857, Wheaton purchased his interests in the local flour mill and other properties—an act that may have economically saved the town. For some time, Wheaton's Northfield Mills produced "choice family flour."