When Carlson was an infant, his father contracted tuberculosis, and also later suffered from arthritis of the spine (a common, age-related disease). When Olaf moved the family to Mexico for a seven-month period in 1910, in hopes of gaining riches through what Carlson described as "a crazy American land colonization scheme," Ellen contracted malaria. Because of his parents' illnesses, and the resulting poverty, Carlson worked to support his family from an early age; he began working odd jobs for money when he was eight. By the time he was thirteen, he would work for two or three hours before going to school, then go back to work after classes. By the time Carlson was in high school, he was his family's principal provider. His mother died of tuberculosis when he was 17, and his father died when Carlson was 27.