The family moved to Virginia City, Nevada and it was whilst living here that Samuel began to be unfaithful to Fanny. In 1866 he headed off gold prospecting in the Coeur d'Alene Mountains, and Fanny and her daughter journeyed to San Francisco. There was a rumour that Sam had been killed by a grizzly bear, but he returned to the family safe, and a second child Samuel Lloyd was born in 1868. But Samuel continued philandering and Fanny returned to Indianapolis.