In 1754, having succeeded to the prosperous family estates (including Anstey Hall in Trumpington), Anstey withdrew from the university. Two years later, he married Ann Calvert (1732–1812), daughter of Felix Calvert and the sister of his friend John Calvert of Albury Hall, Hertfordshire. For a considerable time Anstey lived the life of a country squire, cultivating letters as well as his estates, but publishing little of any note for many years. His family grew to include thirteen children, eight of whom survived him.