Danby Wiske was mentioned in the Domesday Book as "Danebi". The lands were the property of "Kofse" at the time of the Norman conquest. After 1086 the manor was granted to "Landric of Hornby". There were three households, five ploughlands and six acres of meadow. During the late 13th century and early 14th century, the manor was the subject of dispute. Originally it had passed to the lords of Richmond, who had granted hereditary lordship to "Geoffrey le Scrope". When his son Henry inherited the manor the Crown took the title for a short time until it was returned to Henry. However the Crown retook possession upon learning the conditions of inheritance and granted it back to the lords of Richmond in 1342. The manor was leased to Robert Dawe and Edward Thurland in 1602, but in 1616 King James I granted the manor of Danby Wiske to George Calvert, Lord Baltimore, founder of the state of Maryland in the United States. The Calverts sold the manor in 1701 after which it changed hands a couple of times until in 1718 it was bought by Sir Hugh Smithson. It was sold to Jonathon Wharton in 1765. By 1857, the manor was in the possession of the Venerable Archdeacon Cust