By his marriage with Elizabeth Tresham, daughter of Sir Thomas Tresham, he had six children: three sons, three daughters. The eldest son, Henry, succeeded him as 14th Baron Morley and 5th Baron Monteagle. These baronies fell into abeyance when Henry's son Thomas died in about 1686. His oldest daughter, Frances, was a nun; the second, Catherine, married John Savage, 2nd Earl Rivers; and the youngest, Elizabeth, married Edward Cranfield, and was the mother of Edward Cranfield. Through Catherine, he was the 5x great-grandfather of Alfred, Lord Tennyson.