O'Grady moved to in 2004 to concentrate on races such as Paris–Roubaix and the Tour of Flanders. After a start fraught with injuries and doping allegations in his team, he won two stages and the points classification in the Critérium du Dauphiné Libéré. He won a stage in the 2004 Tour de France and spent a few days in the green jersey. He won the UCI Road World Cup race, HEW Cyclassics. He topped his victories by winning an Olympic gold medal in the madison cycling with Graeme Brown.