In 1951, he married Alvilde, Viscountess Chaplin, née Bridges, a prominent gardening and landscape expert.[1] Alvilde Lees-Milne died in 1994. Both Lees-Milne and Alvilde were bisexual, and Alvilde is reputed to have had lesbian affairs with Vita Sackville-West and Winnaretta Singer, among others.[4]