Mary FitzClarence was born in Bushy House as the fourth child and second daughter of the then Prince William, Duke of Clarence, and his companion Dorothea Jordan. She was "a fine looking, brown girl with a pleasant countenance and manners". In 1820, her younger sister Elizabeth was courted by Charles Richard Fox, the eldest but illegitimate son of Lord and Lady Holland.