Dale Gorman and Paul Smyth had shots blocked before Rauno Sappinen put the hosts in front on 50 minutes.
Sheffield United teenager David Parkhouse levelled with a composed back-post finish on his debut.
NI captain Donnelly slotted home the spot-kick in the fourth minute of added time after a foul on Parkhouse.
It was the perfect start to the campaign as U21 clinched a deserved Group Two success in the first game in charge for manager Ian Baraclough.
Northern Ireland's Under-21s have never progressed to a European Championship finals and finished last in their previous qualifying group after picking up only two points from 10 matches.
Baraclough's Under-21 side will also face Albania, Iceland, Slovakia and Spain in their group.
Liam Donnelly's injury-time penalty gave Northern Ireland U21s a dramatic 2-1 victory over Estonia in their opening Euro 2019 qualifier in Tallinn.