At the end of the wars, the Brownes were able to show that their heir was too young to have taken part in the rebellion and they retained the lands. By about 1688, they had erected a mansion house near the castle, but their adherence to King James II of England after the Glorious Revolution caused them to be exiled. The castle became a military barracks, which remained so until early in the 19th century. The Brownes did not return to live at Ross but built Kenmare House near Killarney.