A survey by the Metropolitan Police Federation will consult all its members on their views about equipping officers with weapons.
An extra 600 firearms officers in London have been pledged following the Paris terrorist attacks.
Scotland Yard said it did not support the routine arming of officers.
Does the presence of armed police reassure people?
Ken Marsh, chairman of the Federation, which represents 32,000 officers in London, said it was "only fair" to ask police what they wanted amid a "constant" terror threat.
Questions being put to officers in the survey include:
"We're not an armed force, we never have been" said Mr Marsh.
"But the terrorism threat in London is constant and our officers must be vigilant and be able to deal swiftly with any scenarios we face.
The threat level for international terrorism in the UK has been severe - meaning an attack is highly likely - since August 2014.
Mr Marsh added arming officers could have a "lot of implications" - including harder fitness tests for officers.
"This is not about just giving someone a gun and saying 'get out there'" he said "there's a huge amount of work that would need to be done".
Nationally, the number of armed police is being increased by 1,500.
Unlike most other countries most officers in the UK are unarmed.
2,139
authorised firearms officers
6.7%
of the total police ranks
3,974 police firearms operations between March 2015 and March 2016
7 incidents where police firearms were discharged in 2015-16
A Scotland Yard spokesman said: "We are proud to maintain the tradition that police in this country are not routinely armed.
"Twice in recent years the Met has increased the number of specially-trained officers who carry Taser, and this is kept under constant review."
Thousands of Met Police officers are to be asked if they want to be routinely armed with a gun or Taser.