She was born Mary Carlisle in Richmond, Virginia, at the home of her maternal grandparents. She would live most of her life in the Washington, D.C., area. Her family was extremely wealthy; her father, Calderon Carlisle, was a well known and successful lawyer. This privilege helped her get piano lessons with Hermione Seron, an accomplished pianist. By the time she was 18, she was performing publicly and was accepted into Baltimore's Peabody Institute. It was there that she began studying with Richard Burmeister, who taught her to be quite accomplished on the piano. She also studied composition with Gustav Strube, Ernest Hutcheson, and Harold Randolph (1861–1927), and in 1933 went to Paris to study with the famous French pianist Nadia Boulanger.