He was educated at Sydney Technical High School (despite winning a bursary award to Sydney Boys High School) and the University of Sydney, where he studied English and psychology from 1957 to 1960, and became associated with the Sydney Push, a libertarian intellectual subculture. At the university, he contributed to the student newspaper, "Honi Soit", and directed the annual students' union revue. He graduated with a Bachelor of Arts with Honours in English in 1961. After graduating, James worked for a year as an assistant editor for the magazine page at "The Sydney Morning Herald".