Harris was born in Jackson, Tennessee. As a child, he moved with his family to Rich, Mississippi. He was married to Harriet Anne Haley from 1961 to 1967. They had a daughter, Elizabeth Anne. His long-term domestic partner is Pace Barnes. Harris did not give any interviews between 1976 and 2019. 