In 1913, the family moved to Oakley, Idaho, and bought a farm, where they grew and subsisted largely on Idaho potatoes. The farm was not on good land and failed when potato prices fell. The family moved to Salt Lake City, Utah, in 1916, where Gaskell Romney resumed construction work, but the family remained generally poor. In 1917, they moved to Rexburg, Idaho, where Gaskell became a successful home and commercial builder in a growing area due to high World War I commodities prices.