The 27-year-old was detained as part of an intelligence-led operation after a concerned family member is believed to have contacted the police.
He was held on Thursday on suspicion of possession of an offensive weapon and preparing for acts of terrorism.
Mr Ali is thought to be a UK national - but born overseas - and to have gone to school in Tottenham, north London.
He is still being held in custody at a south London police station.
He was known to the police and the security service MI5. Police are still searching addresses linked to him.
Seven years ago, Mr Ali travelled to Gaza as part of an aid convoy.
He was one of 10 aid workers travelling there by sea when their ship was diverted to Athens after a dispute with the captain.
The activists said the ship captain had become irritable over money and held the group against their will.
He was arrested in Parliament Street, at the junction with Parliament Square, following a stop and search as part of an ongoing operation, police said.
They added that knives had been "recovered from him" but said there was "no immediate known threat" as a result of the arrest.
A witness described having seen two knives on the ground, one of which he said was a large bread knife.
Meanwhile, a woman has been shot during a raid on a house in Willesden, north-west London, another house raided in Kent and six people arrested, in an operation to foil what police said was an active terror plot.
Neither operation was said to be linked to the arrest of Mr Ali in Westminster.
By BBC home affairs correspondent Dominic Casciani
Two counter-terrorism operations in the space of a few hours. This is a fast-moving story - new facts are emerging all the time.
The official threat level from international terrorism in the UK is "severe", which means that security chiefs think that attacks are highly likely.
That is obviously a truism given events in Westminster five weeks ago.
Despite these two separate operations in the last 24 hours, officials have not ratcheted up the threat level to the highest status of "critical", meaning an attack is imminent.
That means the police and MI5 are confident that their current intelligence of known threats - and "known" is the operative word here - is good.
Read more from Dominic here
A terror suspect arrested near the Houses of Parliament is Khalid Mohamed Omar Ali, the BBC understands.