O'Brien studied at the University of Edinburgh where he gained a Bachelor of Science degree in chemistry in 1959 (and a Diploma in Education in 1966). His studies for the priesthood were at St Andrew's College, Drygrange, Roxburghshire, and he was ordained priest on 3 April 1965 by his predecessor, Cardinal Gordon Gray. Initially serving as curate at Holy Cross, Edinburgh from 1965 until 1966, he completed his teacher training certificate at Moray House College of Education. From 1966 to 1971, he was employed by Fife County Council as a teacher of mathematics and science; he also served as chaplain to St Columba's Secondary School, initially in Cowdenbeath and then in Dunfermline, while assisting at St Bride's Parish, Cowdenbeath.