In 1830, Aspinwall was married to Anna Lloyd Breck (1812–1885). She was the daughter of Catherine Douce (née Israel) Breck (1789–1864) and George Breck (1784–1869), the brother of U.S. Representatives Samuel Breck and Daniel Breck. Anna was the sister of Rev. James Lloyd Breck, an Episcopal missionary, and Jane Breck, who married Aspinwall's brother John. Together, William and Anna were the parents of:   William Henry Aspinwall died of a myocardial infarction on January 18, 1875 in Manhattan. He was buried at Green-Wood Cemetery in Brooklyn, New York. The town of Aspinwall, Panama (now Colón), founded in 1850 and named after Aspinwall, on the on the western end of a treacherously marshy islet covered with mangrove trees, known as Manzanillo Island.