Kicked off the force and humiliated, Nestor finds himself drawn to the very neighborhood that ended his career with the Paris police - returning to Chez Moustache, a popular hangout tavern for prostitutes and their pimps. Down on his luck, Nestor befriends Irma La Douce (Shirley MacLaine), a popular prostitute. He also reluctantly accepts, as a confidante, the proprietor of Chez Moustache, a man known only as "Moustache." In a running joke, Moustache (Lou Jacobi), a seemingly ordinary barkeeper, tells of a storied prior life, claiming to have been, among other things, an attorney, a Colonel in the Foreign Legion, and a doctor who worked with Albert Schweitzer in Africa, ending with the repeated line, "But that's another story". After Nestor defends Irma against her abusive pimp, Hippolyte, Nestor moves in with her, and he soon finds himself as Irma's new pimp.