edward swift isham was born in bennington, vermont on january 15, 1836. he was the eldest son of semantha (née swift) isham (1808–1896) and pierpoint isham (1802–1872), later a justice of the vermont supreme court. among his siblings was mary adeline isham, the wife of sartell prentice (their son ezra married alta rockefeller), and henry pierpont isham, a chicago real estate broker and banker.