Higgins was born in Nouméa, New Caledonia, France, to an Australian mother, Joy (Kelly), and American father, John Edward Higgins, one of six sons. Higgins' father enlisted in the army following the attack on Pearl Harbor, and his mother returned to her home in Sydney with Colin and his elder brother. Apart from a brief stint in San Francisco in 1945, Higgins lived in Sydney until 1957, mostly in the suburb of Hunters Hill, attending school at Saint Ignatius' College, Riverview.