While his uncle, Robert E. Lee, was battling Ulysses S. Grant at the North Anna River, Fitz Lee took elements of three cavalry brigades plus the 5th South Carolina Cavalry Regiment (2,500 men and one cannon) on a 40 miles march from Atlee's Station to reach Wilson's Wharf.