Stuart Goddard was born in Marylebone, London. He was the only child of Leslie Alfred Goddard and Betty Kathleen Smith. His father had served in the Royal Air Force and worked as a chauffeur, and his mother was an embroiderer for Norman Hartnell. His home was two rooms in De Walden buildings, St John's Wood. He recalls: "There was no luxury, but there was always food on the table.” He is of partial Romani descent; his maternal grandfather, Walter Albany Smith, was Romanichal. This heritage became a basis for a theme in his later work: a concern for oppressed minorities.