Johnny Rodrigues, chairman of the country's Conservation Task Force, told the BBC that elephant, could have lived for another 15 years.
The unnamed hunter, who was pictured with the corpse, reported paid $60,000 (Â£39,000) for the legal hunt.
It comes months after the controversial death of Cecil the lion.
There was international outrage after Cecil, a 13-year-old lion, was killed during another legal hunt in July, also in Zimbabwe.
Walter Palmer, the US dentist who shot the lion, found out earlier this week that he will not face charges over the shooting.
The death of the elephant also took place during a hunt in Zimbabwe's southern Gonarezhou National Park, Mr Rodrigues said. But he called for such animals to be looked after and used as tourist attractions.
"It's hard to swallow that people are just destroying all the majestic animals that we've got in the country," he said.
"When you see an animal like that, which you haven't seen for years, you should actually put a collar on him and use him as a marketing tool."
The Zimbabwe Association of Safaris Operators (Zaso) said the elephant was the largest seen in Zimbabwe in 100 years. Its tusks, which were so large they almost touched the floor, weighed 120lbs (55kg).
Zaso chairman Emmanuel Fundira said the hunter should have thought twice before shooting.
Mr Palmer was vilified after being identified as the killer of Cecil the lion and he closed his dental practice in Minnesota temporarily after campaigners gathered outside.
Animal rights charities criticised the decision not to prosecute him.
Earlier this week, rangers at Zimbabwe's Hwange National Park, where Cecil was killed, discovered the carcasses of 26 elephants killed by cyanide poisoning.
Poachers are known to poison salt pans with the drug in order to steal the elephants's tusks.
A leading conservationist in Zimbabwe has criticised the killing of a "majestic" 60-year-old elephant by a German hunter, calling it "unethical".