Campbell's property in Kirribilli was used for grazing under lease to Campbell's friend James Milson, hence the name "Milson's Point". Milson's Point is the next point along from Kirribilli point, where the Sydney Harbour Bridge now crosses the harbour. A plan of sub-division was drawn up in the 1840s by Campbell. In 1842, the site where Admiralty House now sits was leased to the Collector of Customs for the Colony, Lieutenant-Colonel (later full Colonel) Gibbes, MLC. Gibbes intended to build a private home on the site. (Since his arrival in the colony in 1834, Gibbes and his family had been living in Henrietta Villa, also known as the Naval Villa, on Sydney's Point Piper, under a leasehold arrangement.)