Robert Greville, 4th Baron Brooke of Warwick married Doddington's granddaughter. His brother inherited the property upon his death in 1676. Francis Greville, great-grandson of the brother, was Earl of Warwick and 8th Baron Brook. The manor was held by the Grevilles, who visited only occasionally, made only minor modifications, and sold it to the Franklen family in the 18th century. Between 1806 and 1808, Sir John Nicholl resided at the manor while his manor was built. Candleston Castle was sold by Richard Franklen to Nicholl in 1830, when it was said to have included a drawing room, dining room, breakfast rooms, four bedrooms and a coach house. It was evacuated in the 19th century, after having operated as a farm house.