 She was the daughter of the former Harriet C. Crosby and Hanson A. Risley, a prominent civil servant who later worked for the Secretary of the Treasury and resided in Washington, D.C. She was the fourth of five children, though her three elder siblings died young.She attended local schools and grew up in the cosmopolitan atmosphere of the capital.Former Secretary of State William Henry Seward, widowed in 1865, took such an interest in Olive Risley beginning in 1868 that Gideon Welles wrote in his diary