The New Zealander takes charge of Scotland for the final time on Saturday, when Italy visit Murrayfield at the end of the Six Nations campaign.
"He's certainly leaving Scotland in a better state than he found it," said Nicol of Cotter's three-year reign.
"He has brought reality and a belief with the improvements he's made. He was so close to a World Cup semi-final."
Cotter will switch to French Top 14 side Montpellier in the summer, with Gregor Townsend moving to Murrayfield from his role as head coach at Glasgow Warriors.
"The team is playing better, the players have developed in his three years and he's done a great job, in my opinion," the former scrum-half told BBC Radio Scotland.
"Let's just ignore what happened at Twickenham last Saturday (the 61-21 defeat to England) because if Scotland win tomorrow this will have been a great Six Nations.
"Potentially, with results elsewhere, we could finish second, which would be a fantastic result.
"Gregor Townsend will be taking over with a really good squad, good age demographic. They are playing really good rugby and they can play even better. I think the future is very bright."
Nicol believes the Scotland players will be motivated to rediscover the form they showed in the home wins against Ireland and Wales and the 22-16 defeat by France in Paris.
"They'll be desperate to go out and prove to themselves, to the coaches, to the Scottish public, that what we saw at Twickenham was not the real Scotland," he said, looking forward to the match against the Azzurri.
"I'll put the England game down to a very bad day at the office, but I expect us to get back to the levels we showed in the first three games, playing with attacking prowess, with a bit of bite in defence and tactically being spot on.
"I expect a different Scotland to what we saw last week."
Vern Cotter has restored belief to the Scotland squad, according to the former national team captain Andy Nicol.