Varina Banks Howell was born in 1826 at Natchez, Mississippi, the daughter of William Burr Howell and Margaret Louisa Kempe. Her father was from a distinguished family in New Jersey: his father Richard Howell served several terms as Governor of New Jersey and died when William was a boy. William inherited little money and used family connections to become a clerk in the Bank of the United States.