A Shelbourne schoolboy player, O'Leary signed for Arsenal as an apprentice in 1973. He soon progressed through the ranks at Highbury, playing in the reserves at the age of 16. He made his debut for Arsenal against Burnley on 16 August 1975, and despite being only 17, went on to make 30 appearances that season. For the next 10 years, he was ever-present in the Arsenal side, playing more than 40 matches a season (except for 1980–81, when he was injured and only played 27).