 He had a younger brother, Bernard.He grew up in relative poverty with his English-born Jewish immigrant parents in the Bronx and in Sea Gate, Brooklyn.He was the great-grandson of the Jewish bare-knuckle pugilist Barney Aaron.