The hosts conceded a try after just 27 seconds and were 13-6 down at the break, Northampton's Harry Mallinder kicking two penalties.
Then Harlequins centre Joe Marchant got onto a grubber kick from Mallinder, who added two more penalties.
Ireland will meet Argentina in the other semi-final.
England's victory meant they finished on 14 points, the best of any team in the pool stages.
They were runners-up in the 2015 tournament to New Zealand but won the previous two competitions.
Elsewhere, Wales raced into an 11-0 lead against reigning champions New Zealand, but the Six Nations Grand Slam winners lost 18-17 to a late penalty from Kiwi centre Jordie Barrett.
It means the Welsh finish third in their group behind the Baby Blacks and Ireland, who made it three wins from three with their 36-7 with over Georgia.
Argentina topped their pool after a 39-20 win over Japan.
Scotland won two pool games for the first time in their history with a 27-19 victory over Italy as they finished second behind England.
The three pool winners plus the best runner-up have qualified for the semi-final proper, while the other eight teams are essentially split into two other sets of semi-finals.
So fifth to eighth and ninth to 12th will become their own mini groups and every team ends in a final position after two more games.
Wales will come up against Scotland and holders New Zealand, who cannot retain their trophy, will face Australia.
Subscribe to the BBC Sport newsletter to get our pick of news, features and video sent to your inbox.
England will face South Africa in the semi-final of the World Rugby Under-20 Championship after beating Australia 17-13 to top pool B.