Born David Neustadt in Haifa, he grew up in Zichron Ya'akov and Rosh Pinna. After completing secondary school in Haifa, Ayalon went in 1933 to study at the recently founded Hebrew University in Jerusalem.