Fry grew up in the village of Booton near Reepham, Norfolk, having moved from Chesham, Buckinghamshire, at an early age, where he attended Chesham Preparatory School. He has an elder brother, Roger, and a younger sister, Joanna. Fry briefly attended Cawston Primary School in Cawston, Norfolk, before going on to Stouts Hill Preparatory School in Uley, Gloucestershire, at the age of seven, and then to Uppingham School, Rutland, where he joined Fircroft house, and was described as a "near-asthmatic genius". He took his O-Levels in the summer of 1972 at the age of 14 and passed all except Physics, but was expelled from Uppingham half a term into the sixth form. He was later dismissed from Paston School, a grant-maintained grammar school that refused to let him progress to study A-Levels.