Falco, a black German shepherd, was taken off active duty by Lincolnshire Police after a Yorkshire terrier named Barbie was bitten earlier this month.
The police dog was also accused of biting a member of the public in a separate incident two days earlier.
The Independent Police Complaints Commission (IPCC) is investigating both incidents.
More on this and other stories from across Lincolnshire on our Live page
The incident involving Barbie happened on a private lane while Falco was with handler Mick Judge. The three-year-old terrier had to be put down because of the extent of her injuries.
Barbie's owner Charles Giermak said the attack on his dog could have been avoided if Falco had been muzzled.
In a statement, Deputy Chief Constable Heather Roach said: "This was an incredibly tough decision to make given the nature of the circumstances and the fact that our dog handlers care a great deal about their dogs, as we do as a force.
"As difficult as it was to take this action, which was carried out with very heavy hearts, it was decided that the best course was to euthanise Falco to ensure this type of incident never happens again.
"An IPCC investigation is still under way so we are limited in what we can say in relation to the exact circumstances until those enquiries have been completed."
A police dog that fatally injured another dog in an unprovoked attack has been destroyed, its force has revealed.