Miller scored his first professional goal on 30 July 2003 in the first leg of the second qualifying round of the UEFA Champions League, coming on as a late substitute to complete a 4–0 win at Lithuania's Kaunas. Two months later in the group stage, he scored against Lyon, and subsequently manager Martin O'Neill offered him a long-term contract, attempting to keep Miller at the club. His first Scottish Premier League goals came in a 5–0 win over Hearts on 18 October. Miller was on the pitch as a substitute when Celtic secured the 2003–04 Scottish Premier League title.