Police believe suicide bomber Salman Abedi, who was born in the city, acted as part of a network.
The 22-year-old was born to Libyan parents who fled their homeland as opponents of Colonel Gaddafi's regime.
Mohamed Abdul Malek, of Manchester's Libya Watch human rights group, said: "The Libyan community is devastated."
"Some even expressed shame at this act," he added.
In total, 22 people were killed in the Manchester Arena attack.
Mr Malek said he could not understand how someone whose family had been welcomed to the UK, and who had been raised in the British educational system, could "blow up people who could have been classmates? He's 22 years old".
"It is shocking really."
And he issued a defiant messsage: "This cowardly act will not divide us. If it did then the terrorists will win.
"I have hopes that we will come out of this, that we will prevail. We will be stronger."
He said the Libyan community in Manchester wanted to do all it could to help and was collecting money for the victims of the bombing.
"It's a token but we are part of this community, we are devastated and we do not accept what this man did."
The BBC has been told that members of the public called the police anti-terrorism hotline about Abedi's extreme and violent views several years ago.
But it is not yet known how officers responded.
The 22nd victim of the attack has been named as 15-year-old Megan Hurley, from Halewood in Merseyside.
The UK terror threat level remains "critical" - meaning another attack could be imminent.
On Friday, police said they had searched an address in St Helens, Merseyside in connection with the attack.
Residents were moved from their homes in Wigan on Thursday night while armed police and a bomb disposal unit searched a house, but they have since been allowed to return.
Libyans living in Manchester have been left "devastated and ashamed" by Monday's terror attack, a community leader has told the BBC.