Lilly went to the Chapin School in New York City, along with Bouvier sister Jacqueline. In 1949, she graduated from Miss Porter's School in Farmington, Connecticut. She attended the college-finishing school Finch College in New York City, but left after one semester to work as a midwife's assistant in West Virginia and later as a volunteer at the Veterans Hospital in The Bronx.