A 4-0 defeat in Ukraine ended remote qualification hopes for Euro 2017, with Scotland gathering just eight points from eight matches.
Gemmill step ups from overseeing the under-17 and under-19 teams for the two remaining campaign games and beyond.
Former Sunderland boss Sbragia quit after 13 months in the role but will stay on at the Scottish FA.
The 60-year-old, who also worked as a coach at Manchester United, succeeded Billy Stark but leaves with Scotland fifth in their six-team group, with no prospect of finishing any higher than fourth.
Former Nottingham Forest and Everton midfielder Gemmill, 45, will be in charge for trips to Iceland and Macedonia next month, with the latter leading Group 3 by a point from France.
"I am very proud to have been given the opportunity to coach the under-21s," he said.
"After working with the under-17s and under-19s, I feel ready for and excited by the challenge ahead.
"I believe we have a number of very talented young players coming through and I will do everything I can to support their development and prepare them for the senior squad."
Ricky Sbragia has resigned as coach of the Scotland Under-21s and will be replaced by Scot Gemmill.