Quinn began in non-league football with Whitchurch Alport and joined Nantwich Town in the 1979 close season from where he moved on to Congleton Town. He stepped up to League football at Swindon Town, John Trollope signing him from non-league Oswestry Town for £10,000 in December 1981, the first of three spells at Swindon's County Ground. He had to wait three months for his debut, coming on as a substitute in a 2–2 draw with Walsall, on 9 March 1982. He made his full debut at the end of the season, forming an attacking partnership with Paul Rideout, in a 3–2 win over Oxford United on 4 May. It did not help Swindon, who were relegated to the Fourth Division at the end of the season, for the first time in their history.