 Each of his three sons served in the conflict in the Union Army.The eldest, John Watts de Peyster Jr., performed duty as an aide-de-camp and artillery commander with the Army of the Potomac and mustered out as a brevet brigadier general; Frederic de Peyster III, was a Colonel and surgeon; while the youngest, Johnston de Peyster, was a second lieutenant in charge of a battery of artillery credited with hoisting the first Union flag over the Confederate capitol of Richmond, Virginia, after its fall.The career militia officer had always suffered from poor health, and turned down a commission as a colonel of cavalry offered to him by New York Senator Ira Harris in June 1863 on behalf of Generals Joseph Hooker and Alfred Pleasonton, who may have had an eye towards de Peyster's social connections.