His eldest brother was sent to Eton College, but Chadwick was educated at Tonbridge School from 1929 to 1935. He was school captain and captain of the rugby team. He then studied classics at St John's College, Cambridge. He received three Blues in rugby playing as hooker for Cambridge R.U.F.C. in the annual Varsity Match against Oxford University in 1936, 1937 and 1938 (as captain). In 1936, during his first year at Cambridge, he was selected to tour with a British Lions team in their third trip to Argentina. Although no caps were awarded on this tour, Chadwick did play in the one match against the full Argentina side, playing in his favoured position of hooker in a 23–0 victory. The British team won all ten of its matches. During the 1937/38 season, Chadwick played for invitational touring side, the Barbarians.