Farrer was born 1 October 1904, the only son of the three children of Augustine and Evangeline Farrer, in Hampstead, London, England. His father was a Baptist minister and Farrer was brought up in that faith. He went to St Paul's School in London he gained a scholarship to Balliol College, Oxford. Encouraged by his father to value scholarship, he nevertheless found the divisions within the Baptist church dispiriting, and while at Oxford he became an Anglican. Finding his spiritual home at St Barnabas Church in Oxford, his theology and his spirituality became profoundly Anglo-Catholic, although centred on the Book of Common Prayer. After gaining a first in greats, he went up to Cuddesdon Theological College where he trained alongside the future Archbishop of Canterbury, Michael Ramsey.