Bungay was home to several literary figures. Thomas Miller (1731–1804), the bookseller and antiquarian, settled in the village. His publisher son, William Miller (1769–1844), was born there. The author Elizabeth Bonhôte, née Mapes, (1744–1818) was born and grew up there, marrying Daniel Bonhôte and writing the notable book "Bungay Castle", a gothic romance. Bonhôte even once owned Bungay Castle. The Strickland family which, according to the "Canadian Dictionary of Biography", was as prolific as the Brontës, Edgeworths and Trollopes, settled in the village 1802–08. Its daughters included Agnes, a historian.