Haxton was born in Greenville, Mississippi, on December 1, 1950. His parents are Josephine Haxton, whose pen name as a writer is Ellen Douglas, and Kenneth Haxton, who is also a Mississippi writer and musician. Haxton has two older brothers: Richard, who is a musician, and Aryes, who is studying to become a lawyer.