As a child, Dwyer attended St. Patrick's Primary School in Grangetown where he joined the school band as a drummer. As his family was relatively poor, he and his friends would often steal fruit and vegetables from gardens and, on one occasion, he was caught shoplifting by a policeman. He took up numerous sporting activities in his youth, including cricket, rugby and baseball, but was most keen on football, idolising the Leeds United side featuring Billy Bremner, as well as Manchester United striker George Best. Dwyer grew up supporting Cardiff City and began attending matches at the club's home ground, Ninian Park, from a young age. Often unable to afford the entry fee for matches, he and his friends would often sneak into the ground by climbing disused railway sleepers to scale the fencing around the stands.