In 1792 the Glasgow merchant John Wallace, owner of extensive estates in Jamaica with sugar plantations and slaves, bought the Kelly Estate. In 1793 he had a red sandstone mansion called Kelly House built on the hillside up from the road, looking over the bay (this was later painted white). About this time the Wemyss Bay Hotel was built on the east side of the main road, near the junction to the road serving the villas; a building is shown there on Ainslie's map.