In October 1787 the Duke of Rutland died at the Vice-Regal Lodge in Dublin, after a short illness, at the early age of 35. Crabbe assisted at the funeral at Belvoir. The Duchess, anxious to have their former chaplain close by, was able to get Crabbe the two livings of Muston, Leicestershire, and Allington, Lincolnshire, in exchange for his old livings. Crabbe brought his family to Muston in February 1789. His connection with the two livings lasted for over 25 years, but during 13 of these years he was a non-resident. He stayed three years at Muston. Another son, Edmund, was born in 1790. In 1792, through the death of one of Sarah's relations and soon after of her older sister, the Crabbe family came into possession of an estate in Parham in Suffolk, which removed all of their financial worries. Crabbe soon moved his family to this estate. Their son William was born the same year.