 Over the porch doorway is the date 1635 with the motto, "Nisi Dominus aedificet Frustra" (the Lord never builds in vain).The poet and politician Sir Benjamin Rudyerd lived in the house and died there in 1658.In the eighteenth century, West Woodhay House was owned by William Sloper, MP, followed by his son William Sloper together with, after a notorious lawsuit and her estrangement from her husband, the soprano Susannah Cibber 

The house passed into the hands of the Cole family and was inherited by the notorious prankster Horace de Vere Cole on the death of his grandmother in 1906.