Georges Head fort was built in 1871 after the British Army decided to leave Australia in 1870. When they left the responsibility for protecting Sydney and Sydney Harbour was now in the hands of rich colonies like New South Wales and Victoria who now had to organise its own defences.