There he met and married Margaret Louisa Kempe (1806–1867), born in Prince William County, Virginia. Her wealthy planter family had moved to Mississippi before 1816.[1] She was the daughter of Colonel Joseph Kempe (sometimes spelled Kemp), a Scots-Irish immigrant from northern Ireland who became a planter and major landowner, and Margaret Graham, born in Prince William County. Margaret Graham was considered illegitimate as her parents, George Graham, a Scots immigrant, and Susanna McAllister (1783–1816) of Virginia, never officially married.[2][3]