A firearms officer has been arrested over the fatal shooting of Jermaine Baker on 11 December.
The 28-year-old was shot dead during an alleged attempt to spring two convicts from a prison van in Wood Green.
Met Police Federation chair Ken Marsh, said a criminal charge would have "massive ramifications" for the force.
The Independent Police Complaints Commission (IPCC) has opened a homicide inquiry following the shooting.
But Mr Marsh said: "If the officer in question was charged, my colleagues would consider very, very carefully if they would still carry blue permits and carry firearms.
"We are talking about police officers who carry firearms by choice and are looking to be backed up by the government for doing this.
"Otherwise, if they are out in the streets in a Paris-style scenario, they don't want to think if they are going to be prosecuted."
Armed police officers could refuse to carry weapons if a colleague is charged over the shooting of a man in London, the Police Federation has warned.