Alison Hernandez, Devon and Cornwall's police and crime commissioner, was on a BBC Radio Cornwall phone-in when a caller suggested the idea.
Ms Hernandez said she would "really be interested" in exploring the issue.
The force said armed civilians would risk being caught in the crossfire.
Deputy Chief Constable Paul Netherton said it was "definitely an emphatic 'no'" that the people should be arming themselves against such a threat.
The caller, named Sara, said she was an "ex-registered firearms dealer" and gun owner living in Bude.
She said: "If there should ever be a terrorist attack close to the town centre... what happens if people like I and others try to defend themselves using those guns?"
Ms Hernandez replied: "This could be some of our solution..." before being interrupted by presenter Laurence Reed, challenging her on whether she was advocating vigilantes.
Ms Hernandez replied: "I'm just saying... let's officially have a look at that and see what would be the implications of it.
"We work with businesses to keep our communities safe. I'd really be interested in exploring that with the chief constable.
"If your community was coming under attack, people would do all sorts of things to try to save their community without even thinking.
"People will do incredible things without thinking to protect themselves and the community around them."
After the phone-in, she told the BBC: "I think everybody should be thinking about what part they can play in a terror attack."
However, DCC Netherton said: "A marauding gunman... is probably the most challenging situation the police service is facing and our response to that is by highly professional officers as well as firearms officers and special forces and we would not want to public to be involved in that situation, especially with firearms.
"As was seen in London, we responded immediately and we don't have time to find out if you're an innocent member of the public or a terrorist.
"If you're carrying a gun we will deal with you and deal with you immediately."
He urged anyone caught up in a terrorist attack instead to "run, hide and inform us".
A police chief has found herself at odds with her own force by saying she would look into whether gun owners could act as armed civilians in the event of a terror attack.