In the summer of 1990 they enjoyed a visit to Martha's Vineyard and returned the following year. They purchased an 1891 eight-bedroom Queen Anne house in Oak Bluffs. They bought and lived in a nearby house while initiating redesign of the main house, which had originally been built by Philip Corbin. It was completed in 1994. "My children are half-black, and we thought Oak Bluffs would give them an opportunity to summer around other kids like them," Norton said in a 2007 interview with Laura D. Roosevelt for "Martha's Vineyard Magazine," alluding to Oak Bluff's reputation as a popular summer spot among upper-class blacks.