 His brother-in-law, Nathaniel G. Taylor (married to his sister, Emaline), was a Whig congressman and Unionist.Taylor's sons (Haynes's nephews) included Alfred A. Taylor (1848–1931) and Robert Love Taylor (1850–1912), who would both serve as Governor of Tennessee.During the Civil War, Haynes's father, David, protected the family's property by telling Union soldiers he was the father-in-law Nathaniel G. Taylor, while telling Confederate soldiers he was the father of Landon Carter Haynes.