 Hart was born in Harlem, New York City, the elder of two sons, to Jewish immigrant parents, Max M. and Frieda (Isenberg) Hart, of German background.Through his mother, he was a great-grandnephew of the German poet Heinrich Heine.His father, a business promoter, sent Hart and his brother to private schools.