 D'Avigdor-Goldsmid was born on 19 December 1912 and was the younger son of Sir Osmond d'Avigdor-Goldsmid, 1st Baronet.He was educated at Harrow School.Having attended the Royal Military College, Sandhurst, d'Avigdor-Goldsmid was commissioned into the 4th/7th Royal Dragoon Guards on 1 September 1932 as a second lieutenant.