Born in Swansea, Ford was the second child of Trevor and Daisy Ford. His father, Trevor Sr, had been a physical training instructor during the First World War before returning to South Wales to work as a delivery van driver and later at a local cinema, eventually being promoted to manager. Ford was encouraged into football by his father who would buy him a new football and boots for each birthday. He grew up in the Townhill district of the city and attended Powys Avenue School as a child, captaining the school's football side as a centre-half.