McGlynn ended his ten-year association with Hearts when he was appointed Raith Rovers manager on 20 November 2006. Rovers' second season in the Scottish Second Division ended in disappointment, finishing third and beaten in the promotion play-off semi-finals against Stirling Albion. In 2007–08, McGlynn's first full season, Raith again finished third and fell at the first play-off hurdle, losing to Airdrie United. In 2008–09, however, McGlynn guided Raith to the Second Division title and subsequent promotion back to the First Division.