Pilrig House, built in 1638 for Gilbert Kirkwood appears to have been built at the fort's SW corner. Stonework in the basement walls suggest that the remains of a peel tower, from which the name Pilrig may derive, are incorporated in the house. The Balfour family were involved financially in the failed Darién expedition to colonise Panama in the late 17th century. As part of the Treaty of Union, landed investors were reimbursed in full in 1707. The Balfour family profited from this and became local entrepreneurs, running, amongst other things, a local stagecoach service. Robert Balfour bought Pilrig House in 1709 with the proceeds, and remodelled the house at that time. The house and estate remained in their possession until the 20th century, their name being recalled by Balfour Street, which links the park to Leith Walk.