When he did move it was not with sufficient confidence or ability on his part to manoeuvre his troops." The fighting on May 3, 1863, was some of the most furious anywhere in the civil war. The loss of 21,357 men that day in the three battles, divided equally between the two armies, ranks the fighting only behind the Battle of Antietam as the bloodiest day of war in American history. On the evening of May 3 and all day May 4, Hooker remained in his defenses north of Chancellorsville. Lee observed that Hooker was threatening no offensive action, so felt comfortable ordering Anderson's division to join the battle against Sedgwick. He sent orders to Early and McLaws to cooperate in a joint attack, but the orders reached his subordinates after dark, so the attack was planned for May 4. By this time Sedgwick had placed his divisions into a strong defensive position with its flanks anchored on the Rappahannock, three sides of a rectangle extending south of the Plank Road. Early's plan was to drive the Union troops off Marye's Heights and the other high ground west of Fredericksburg. Lee ordered McLaws to engage from the west "to prevent [the enemy] concentrating on General Early." Early reoccupied Marye's Heights on the morning of May 4, cutting Sedgwick off from the town. However, McLaws was reluctant to take any action. Before noon, Lee arrived with Anderson's division, giving him a total of 21,000 men, slightly outnumbering Sedgwick. Despite Lee's presence, McLaws continued his passive role and Anderson's men took a few hours to get into position, a situation that frustrated and angered both Early and Lee, who had been planning on a concentrated assault from three directions. The attack finally began around 6 p.m. Two of Early's brigades (under Brig. Gens. Harry T. Hays and Robert F. Hoke) pushed back Sedgwick's left-center across the Plank Road, but Anderson's effort was a slight one and McLaws once again contributed nothing. Throughout the day on May 4, Hooker provided no assistance or useful guidance to Sedgwick, and Sedgwick thought about little else than protecting his line of retreat. Sedgwick withdrew across the Rappahannock at Banks's Ford during the pre-dawn hours of May 5. When he learned that Sedgwick had retreated back over the river, Hooker felt he was out of options to save the campaign.