It is thought that there was a castle at Clitheroe in 1102, as Robert de Lacy granted lands formerly the property of Orme le Engleis, within the baillie and below, to Ralph le Rous. A charter from 1122 also mentions the castle's chapel. In the summer of 1138, a Scottish force under William fitz Duncan harried the area, defeating an English force at the Battle of Clitheroe. Although the castle is not mentioned in the known accounts of the battle, it may have been the reason for the battle's location.