The village started in 1801 with the building of a small harbour on Kames Bay. Lord Bannatyne of Kames Castle, at the head of the bay, planned the village in an attempt to rival Rothesay. Initially known as "Kamesburgh", by the mid-19th century, steamers were calling there regularly. In 1860 the Marquess of Bute purchased this part of the island and renamed the village Port Bannatyne in honour of the long historical association of the Bannatyne family with the area. Boat building became an important local industry.