Anna's father was Colonel William Smith, son of Henry Smith and grandson of Colonel William Smith, a justice of the supreme court established in New York in 1691.[2]:503 He was clerk of Suffolk County, New York and judge of the Common Pleas court for the county for several years before the American Revolution.[2]:503 Anna's mother was Margaret Lloyd Smith, daughter of Henry Lloyd of Lloyd's Neck.[2]:503