Vardis Fisher was born on March 31, 1895, in Annis, Idaho, near present-day Rigby. His family moved to an isolated home in 1905. Fisher saw livestock raising firsthand, including bull castration, de-horning cattle, and butchering. He read many books in the Rigby library. His family was "nominally Mormon," according to Louie Attebery, professor at the College of Idaho, in "A Literary History of the American West", but in "Mormons and Popular Culture", Mormon literature scholar Michael Austin states that Fisher was raised by "strict Mormon parents," though the remoteness of the Fisher home prevented any contact with a Mormon community. Fisher was not officially baptized into the Church of Jesus Christ of Latter-day Saints until he was 20 years old, and he abandoned the church for good shortly thereafter, though his mother, sister, and children all remained Mormon. "Vardis Fisher was a religious unbeliever," Austin writes, "but Mormonism was the religion that he didn't believe in."