Gundogan, 26, injured his right knee in City's 2-0 Premier League win over Watford on Wednesday.
"It's so, so difficult for him, and so sad for him - we're going to miss him a lot," said Guardiola, who put no timeframe on the German's return.
It is the second serious knee injury that Gundogan has suffered in 2016.
The Germany midfielder, who joined City from Borussia Dortmund in a £20m deal in June, was sidelined for almost five months and missed his country's Euro 2016 finals campaign after dislocating his right kneecap in May.
Gundogan has scored five goals in his 16 appearances for the club since making his debut in the 4-0 Champions League win over Borussia Monchengladbach on 14 September.
Manchester City midfielder Ilkay Gundogan needs surgery on torn cruciate knee ligaments, manager Pep Guardiola has confirmed.