Richard Graves' mother, Jane Ryder (1719–1810), was the daughter of Rev. Thomas Ryder (1683–1747), of Mitchelstown Rectory, County Cork, nephew and heir of the Rt. Rev. Henry Ryder (died 1695), Bishop of Killaloe, Treasurer of Cork and grandson of Bishop John Ryder. Mrs Graves' mother, Martha Badham, was a first cousin of Sir Standish Hartstonge and Sir Matthew Deane. Richard Graves's father was the rector of seven parishes at once. Five of his seven parishes, including Kilfannan, were given to him by his friend, fellow scholar, and kinsman of his both him and wife, John Boyle, 5th Earl of Cork. Mrs Graves' uncle, Brettridge Badham, had married a sister of Henry Boyle, 1st Earl of Shannon, first cousins of the 5th Earl's father, Lord Orrey.