Scharff was born on December 16, 1907, in Rastenburg, East Prussia (now Kętrzyn, Poland), to Hans Hermann Scharf and Else Scharf (née Jahn) and was the second of three sons, the elder being Eberhardt and the younger, Wolfgang, who died in his teens.
Scharff was born on December 16, 1907, in Rastenburg, East Prussia (now Kętrzyn, Poland), to Hans Hermann Scharf and Else Scharf (née Jahn). He is the second of three sons: Eberhardt is older than him, while Wolfgang, his younger brother, passed away during adolescence.