On the strength of these preferments and a promise of future assistance from the Duke, Crabbe and Sarah Elmy were married in December 1783, in the parish church of Beccles, where Miss Elmy's mother lived, and a few weeks later went to live together at Belvoir Castle. In 1784 the Duke of Rutland became Lord Lieutenant of Ireland. It was decided that Crabbe was not to be on the Duke's staff in Ireland, though the two men parted as close friends. The young couple stayed on at Belvoir for nearly another eighteen months before Crabbe accepted a vacant curacy in the neighbourhood, that of Stathern in Leicestershire, where Crabbe and his wife moved in 1785. A child had been born to them at Belvoir, dying only hours after birth. During the following four years at Stathern they had three other children; two sons, George and John, in 1785 and 1787, and a daughter in 1789, who died in infancy. Crabbe later told his children that his four years at Stathern were the happiest of his life.