 Willoughby was appointed the King's Lieutenant in Lower Normandy before February 1432, where he had mixed success, suffering defeats by the French at Vivoin and the siege of Saint-Céneri-le-Gérei in 1432, but capturing St. Valéry after a siege in July–August 1433.He was Captain of Bayeux in 1433, and of Pont de l'Arche about 1434.In July 1435 he raised a force of 2000 men in England, and with Lords Talbot and Scales besieged Saint-Denis, capturing it in October.