Despite his fine all-round cricket game he was rarely considered for One Day Internationals and in 1981 he was controversially left out of the Ashes squad which toured England (Stuart MacGill, Yardley and fellow West Australian Bruce Reid are the only three bowlers to take 100 wickets for Australia without playing a Test there). Yardley participated in Australia's first tour of Sri Lanka in 1983 and took seven wickets, including a five-wicket haul, in what proved to be his final Test.