Arthur Sullivan visited Grim's Dyke with his nephew, Herbert, on 27 May 1893. They stayed at the house for three days while Sullivan worked with Gilbert to finalise "Utopia, Limited". At the house, Gilbert had a vinery, apiary, orchards and a farm, with Jersey cattle, horses, pigs and fowl. He also kept a variety of exotic animals including monkeys, lemurs, a lynx and many others. Lady Gilbert designed the 30 acres of ornamental gardens at the house, which remain as the hotel grounds. After Gilbert's death in 1911, Lady Gilbert and the Gilberts' companion, Nancy McIntosh, continued to live there until Lady Gilbert's own death in 1936.