Scott McBride bundled in Rovers' opener early in the second half and Alistair Love fired home their second.
Gary Fisher teed up McBride for his second before Jamie McCormack headed one back for Stenny.
Results elsewhere ensured Rovers fell short of fourth place in the division as the hosts avoided a relegation play-off by finishing eighth.
Albion Rovers missed out on the promotion play-offs, despite beating Stenhousemuir in Scottish League One.