It was built by the Bissett family in the 13th century on the site of an earlier motte-and-bailey outpost of the Kingdom of Dál Riata. The Bissett family were stripped of their lands in Scotland and fled for their lives to Ireland after Walter de Bisset was accused of the murder of Patrick, Earl of Atholl, at Haddington, East Lothian in 1242. King Henry III of England granted Bisset large possessions in the Barony of Glenarm, Ireland.