Wolters was born into a Catholic family in Coesfeld, Germany on August 3, 1903, the son of an architect who had married the daughter of a master carpenter in the shipbuilding trade. In his privately published memoirs, "Segments of a Life", Wolters described his father as "a serious, conscientious and diligent man, always concerned about the future". Wolters regarded his mother as "a highly practical woman, full of zest for life, who in hard times thought nothing of serving a delicious roast without letting on it was horsemeat". Wolters passed a generally happy childhood, punctuated by the chaos of the war years, and by a childhood illness that resulted in his being taught at home for a year by two priests.