Smith remained with Mrs. Emery until 1963, except for two years spent with her daughter, Mrs. Norman Hanson, who lived across the street, and another year in New York when Mrs. Emery was in California. Mrs. Emery later echoed Edith Kohlstaat's comments that Smith was difficult to live with, recalling that she often seemed morose, objected to visits by some of Mrs. Emery's friends, did not get along with the servant, and became annoyed when she was not permitted to use the family car.