 At the outset of the Civil War, Kelly enlisted in the Union Army and saw action as captain of Company E of the 69th New York Infantry at the First Battle of Bull Run.He briefly was a captain in the 16th U.S. Infantry.On September 14, 1861, he was named lieutenant colonel of the 88th New York Infantry and fought in the Irish Brigade's major battles in 1862.