Yudkin was raised in the East End of London in an Orthodox Jewish family that had fled the Russian pogroms of 1905. His father's death left six-year-old Yudkin and his four brothers to be raised by their mother in considerable poverty.