Fussell was born in Pasadena, California on March 22, 1924. He was the second of three children. His father, Paul Fussell (1895–1973), son of a widowed schoolteacher, became a corporate lawyer in Los Angeles with the firm of O’Melveny & Myers. His mother, Wilhma Wilson Sill (1893–1971), was the daughter of a carriage trimmer in Illinois.