Zambia looked like they had the match won at 3-1 in the 89th minute but late goals from Suat Serdar and Jonas Arweiler ensured the game went to extra time.
Shemmy Mayembe was the Zambian hero with a goal early in the second period of the added 30 minutes.
Zambia will now play France or Italy in the last eight on Monday.
Philipp Ochs had opened the scoring for the Germans with a freekick that crashed in off the bar eight minutes before the break.
Emmanuel Banda scored the equaliser for Zambia five minutes into the second-half after he picked up the ball when Enock Mwepu miscued a bicycle kick following a cross by Edward Chilufya.
In the 67th minute Mwepu was again involved as he set up Fashion Sakala to put Zambia ahead with a fierce shot from a narrow angle.
Mwepu then scored himself in the 85 minute after a Zambian counterattack with Patson Daka putting through the final ball.
The Germans reduced the arrears in the 89th minute when Suat Serdar scored a back-heel past keeper Mangani Banda.The Germans levelled the match in injury time with substitute Jonas Arweiler sending a low drive beyond Banda to send the game into extra time.
Zesco United defender Shemmy Mayembe was the unlikely match winner when he found space in the box and pounced on a pass from Conlyde Luchanga in 107th minute.
African champions Zambia came from behind to beat Germany 4-3 after extra-time and advance to the quarter-finals of the Under-20 World Cup in South Korea.