Miller joined Celtic in 1997 as a youth player, and made his professional league debut against Dundee United on 21 May 2000, as a 77th-minute substitute for fellow debutant Ryan McCann. His UEFA Cup debut came against Luxembourg-based club Jeunesse Esch on 24 August 2000 in the second leg of the qualifying round. He came close to scoring in the 64th minute of the 7–0 win (11–0 aggregate). During his time at Celtic Park, he had been on a six-month loan spell with Danish football club AGF Aarhus during the 2001–02 season, making 18 appearances without scoring a goal. They considered signing him permanently for £300,000.