 His grandfather was William Oliver Wheeler, who fought with the 12th Royal Lancers against Napoleon in Portugal and Spain, and later became mayor of Kilkenny.Wheeler's father, Edward Oliver Wheeler, was a captain in the Kilkenny Fusiliers.He is the father of Sir Edward Oliver Wheeler, who participated in the first topographical survey of Mount Everest in 1921, and as Brigadier in the British Army was appointed Surveyor General of India in 1941.