L'Hermitte's rising star came to a halt, however, when Truguet was dismissed after speaking against the rise of the Empire, and for one year l'Hermite was left without a command. He was appointed Officer of the Legion of Honour at the founding of the Order in late 1805.
L'Hermitte's career took an abrupt turn when Truguet was let go after speaking out against the Empire's growing power, leaving l'Hermite without a position for a year. He was subsequently honored with the Legion of Honour at its establishment in 1805.