The 30-year-old Belgian was taken off during Saturday's 2-1 victory at Crystal Palace after being concussed, but also has knee-ligament damage.
Guardiola said Kompany is attending a Spanish hospital for treatment, adding: "He's sad but he is in the best hands."
Kompany, who has started two league matches this season, has already had calf, groin and thigh injuries in 2016.
It is his 35th separate injury since joining City from German club Hamburg in 2008.
"We have to try to help him week after week to see if he can play more often," added Guardiola, whose side travel to Borussia Monchengladbach in the Champions League on Wednesday.
"He could be out for weeks. He's sad and I'm sad. It's a long time he doesn't play."
Kompany has played 303 times for City, but the 22 appearances he made in 2015-16 is the fewest he has made in a season for City.
He has started 15 matches in all competitions this year and been taken off in five of them, with one other appearance from the bench.
When he returned in October from a six-month lay-off he said he would be "honest" with the club's medical staff and listen "when they made a decision".
Manchester City captain Vincent Kompany could be out for "weeks" after injuring his knee, says manager Pep Guardiola.