McKinnon, 45, held talks with former club Dundee United about their managerial vacancy earlier this week.
Mixu Paatelainen left United on 4 May after their relegation from the Premiership had been confirmed.
Raith missed out on promotion to the top flight via the play-offs and McKinnon had a year left to run on his contract at Stark's Park.
The Kirkcaldy club said on their website: "The search for a new manager has already begun."
And chief executive Eric Drysdale said: "Everyone at the club is naturally very disappointed that Ray has reached this decision, and we certainly did everything in our power, and then more, to try to retain his services but he leaves Raith Rovers with thanks and best wishes for the future.
"Ray has built a terrific squad of players with tremendous team spirit, and that will be his legacy."
McKinnon, who had two spells as a player at both United and Raith, joined Rovers as manager last year after three years in charge of Brechin City.
He led Raith to the Premiership play-off quarter-final but they lost 2-1 on aggregate to Hibernian.
United were bottom of the Premiership for most of the season, with Paatelainen replacing Jackie McNamara as manager in October.
Defeat by city rivals Dundee on 2 May ensured United would finish at the foot of the table and start next season in the Championship.
Ray McKinnon has resigned as manager of Raith Rovers with immediate effect, the club has confirmed.