After Whittier's death in 1892, his niece, Lizzy Pickard, was not interested in maintaining the house. The Whittier Home Association was established in 1898 as a vehicle to preserve the house and the Whittier legacy. Lizzie's son, Greenleaf Pickard, in 1904 decided to live in the house, but made a series of additions in part to preserve the original portion used by Whittier. The west side of the house was also raised to a full two stories, and the kitchen ell was replaced by a two-story addition. The present front porch is believed to date to this time period as well. The old kitchen ell was separated from the house and is now a small outbuilding.