 Cooper was born in New York City on October 26, 1824.He was the son of Peter Cooper (1791–1883) and Sarah Raynor (née Bedell) Cooper (1793–1869).His sister Amelia Cooper (1830–1912) was married to Abram Stevens Hewitt (1822–1903), who also served as mayor of New York from 1887 to 1888.