The Gresham family had been settled in the Norfolk village of Gresham since at least the late 14th century. Richard Gresham's grandfather, James Gresham, moved to Holt, Norfolk, about three miles from Gresham, where in the mid-fifteenth century he built a manor house in the centre of the small town. Richard Gresham, born about 1485 at Holt, was probably the third of four surviving sons of James Gresham's son, John Gresham, by his marriage to Alice Blythe of Long Stratton, Norfolk.