Young Ailes grew up in a Japanese style house that Léonie had constructed in Chigasaki, a seaside town near Yokohama. Ailes had close Japanese childhood friends, spoke Japanese as well as English and identified with Japan before she returned to the United States in 1920, at age 8. When Ailes and her mother returned to America, they lived first in San Francisco and then moved to New York City. Léonie was a great believer in progressive education and sent Ailes to the Ethical Culture Society elementary school, founded in 1876 by Felix Adler. Léonie herself had attended the predecessor to the Ethical Culture Society elementary school when it was called the Workingman's School. For high school, Léonie chose the Cherry Lawn School in Connecticut for her daughter. It was a boarding school that was known for its progressive, coeducational program. The director and founder of the school was Dr. Fred Goldfrank, who was related to one of the founders of the Ethical Culture Society. Ailes greatly enjoyed her time there and formed several friendships that she maintained for the rest of her life.