The structure was built in 1817, making it the 12th oldest surviving building in Toronto and the oldest remaining brick house. It was built for D'Arcy Boulton (1785–1846), a son of G. D'Arcy Boulton. He was one of the town's leading citizens and part of the powerful Boulton family, which played an important role in the Family Compact. Originally, it was considerably west of the town of York, but over time, the town grew and Boulton sold his considerable land holdings surrounding the manor for a large profit.