Three women aged between 60 and 82 were hit by bullets and bullet fragments after two officers fired at the man.
The 23-year-old man, who allegedly approached officers with the knife, was also injured.
An independent team will now investigate the circumstances surrounding Thursday's incident.
Police were called to an organic food market outside Westfield Shopping Centre at Hornsby, in Sydney's north, just before midday after shoppers noticed the man acting strangely.
New South Wales Police Assistant Commissioner Dennis Clifford said the man was holding a large carving knife and confronted officers when they approached him.
He said two officers fired shots at the man, who was wounded several times.
"Unfortunately some bystanders were also injured, with either bullet or fragment wounds," Mr Clifford said.
"The man is known to us. I understand he was reported missing from a psychiatric centre near here yesterday and police were making efforts to try and locate him."
Mr Clifford said it was understood that Tasers, batons and sprays were available to the officers.
"In a life-and-death situation the police had to make a very critical decision on the action that they took," he said.
All the injured bystanders are in hospital in a stable condition.
Witnesses told Australian media of a chaotic scene after police shot the "zombie-like" young man, who was "babbling incoherently" before the incident.
He also remains in hospital, under police guard.
Four people have been injured when police opened fire on a man holding a knife at a busy shopping centre in Sydney, Australia.