Keepers found Vince, a four-year-old white rhino, in his enclosure at Thoiry Zoo on Tuesday morning.
One of his horns had been hacked off with a chainsaw, police said.
The African rhino's horn commands high prices on the black market, with about 100 killed every month in the wild.
However, this is thought to be the first time poachers have targeted a rhino living in a European zoo.
Vince was shot three times in the head after poachers forced their way into the zoo overnight. They then took the horn - a kilo of which could fetch as much as $60,000 (Â£49,300).
But the poachers appear to have been disturbed as his second - partially sawn - horn was left behind.
The zoo's two other white rhinos, 37-year-old Gracie and five-year-old Bruno, "escaped the massacre", the Thoiry Zoo revealed.
Bruno and Vince arrived at the zoo together in 2015. Vince, who was born in a zoo in the Netherlands, was one of 250 rhinos in European zoos who were part of a breeding programme.
The white rhino is a conservation success, having been brought from the edge of extinction in the late 19th Century to a population of about 20,000 animals.
However, poaching has risen over the past few years, as demand from markets like Vietnam - where rhino horn is thought to have aphrodisiac properties - increases.
France outlawed the trade in ivory and horns last year.
A rhino has been shot dead by poachers at a zoo in France in what is believed to be the first such incident in Europe.