When Virginia was about 8 years old, her family moved to San Diego, California, as the New England winters were hard on her mother's health. Her father, close to his retirement in 1921 after 40 years at MIT, took a leave of absence. A year later the family settled 450 miles north in Carmel-by-the-Sea, then a small, artistic community. Burton and her sister took dance and art lessons, performing in local productions. Her parents divorced in 1925, and her father returned to Boston.