Scharff was born on December 16, 1907, in Rastenburg, East Prussia (now Kętrzyn, Poland), to Hans Hermann Scharf and Else Scharf (née Jahn) and was the second of three sons, the elder being Eberhardt and the younger, Wolfgang, who died in his teens. Hanns Scharff added an extra ‘f’ to the end of his last name as an adult; some ancestors spelled their last names that way.