In 1997, seven men filed lawsuits against him for alleged sexual abuse. Six of the men said they were abused when they were younger and worked at the Chaplain's Pantry. The case never went to court. Smith said the abuse never happened. The "Frugal Gourmet" empire ended with the scandal.