Mainer was born near Weaverville, North Carolina, on a mountain farm in Buncombe County on April 21, 1907. His family was poor during his childhood and they lived in a log cabin in the Blue Ridge Mountains. Mainer credited his father who was, in Mainer's words, "a good singer – real stout voice", as of one of his influences. During his career as a musical artist, Mainer would perform many of the old songs that he had heard from his father.