The house is best known as the home of the dramatist W. S. Gilbert, of the opera partnership Gilbert and Sullivan, who lived and farmed there for the last two decades of his life. He died while attempting to save a girl from drowning in his lake. Lady Gilbert and the Gilberts' ward, Nancy McIntosh, lived there until her death in 1936. The statue of Charles II now found in Soho Square stood on the property from about 1880 to 1938. The house was then used as a rehabilitation centre until 1963.