Yardley played schoolboy cricket at St Peter's, York. A highly talented all-round sportsman, he went to St John's College, Cambridge, and won Blues at cricket, squash, rugby fives and field hockey. In the university matches, he scored 90 in his second year, 101 in his third and was captain for his final year. He made his Yorkshire debut in 1936 and played for the county until 1955, when he retired as a player. He made his Test match debut against South Africa in 1939 and after the Second World War was chosen as vice-captain to Wally Hammond on the 1946—47 tour of Australia where he captained England in the fifth Test. He followed Hammond as skipper in 1947, and captained England intermittently until 1950 when his business commitments allowed. In 1948 he succeeded to the Yorkshire leadership when Brian Sellers resigned. Yardley remained in the position until 1955, during a time when Yorkshire had several difficult players in their dressing room. Under Yardley, Yorkshire were joint champions in 1949 but subsequently on a number of occasions, too often for the liking of supporters, finished second to Surrey in the County Championship. He served as a Test match selector between 1951 and 1954, acting as chairman of selectors in 1952. He was President of Yorkshire C.C.C. from 1981 to 1983, when he resigned after becoming involved in controversy over the decision to release Geoffrey Boycott in 1983. He died after a stroke in 1989.