Goodfellow was sacked soon after and replaced by former Cardiff player Alan Durban but results showed little improvement and player morale was low. On 17 February 1985, Cardiff suffered a 4–1 home defeat to Notts County and a furious Dwyer returned to the dressing room after the match and lambasted several of his teammates after believing that they had given up during the game. Durban voiced his displeasure with Dwyer over the incident, although the two left the ground on amicable terms. Dwyer was surprised to arrive at Ninian Park two days later to be told that Durban had accepted an offer from Fourth Division side Rochdale to sign Dwyer on loan for the remainder of the season. Cardiff went on to finish the season in 21st position in Dwyer's absence, returning to the Third Division. He finished his career with Cardiff having made 575 appearances in all competitions, a club record in the Football League era and second only to Billy Hardy in the club's history.