hirsch grew up in brooklyn in an affluent jewish family. his father ran a successful chain of bowling alleys. hirsch attended a school of predominantly black students, and he played basketball on the asphalt courts of the bedford–stuyvesant neighborhood. his family moved to the los angeles district of van nuys when he was 14. in his senior year at van nuys high school, the center won all-city co-player of the year honors.