Downs is the daughter of Mixtec cabaret singer Anita Sánchez and Allen Downs, a Scottish/English-American professor of art and cinematographer from Minnesota. She learned to sing at the age of four. She began to perform rancheras songs when she was 10 years old. Lila grew up partly in the Mexican state of Oaxaca, partly in the U.S. state of California as a teenager. She was 15 when she went to Performing Arts Institute of Oaxaca, where she studied music. She lived in Minnesota as an adult. She graduated from the University of Minnesota in voice and anthropology.