In February 1919, Lilian persuaded her husband to take the family back to England for a climate better suited to their ailing daughters. They sailed aboard the SS "Siberia Maru" to San Francisco, where the family stopped to treat Olivia's tonsillitis. After Joan developed pneumonia, Lilian decided to remain with her daughters in California, where they eventually settled in the village of Saratoga, south of San Francisco. Her father abandoned the family and returned to his Japanese housekeeper, who eventually became his second wife.