Though subsequently the territory passed to the Dukes of Burgundy, Hesdin remained one of a handful of French strongholds, until in 1553 Emperor Charles V ordered the utter destruction of the old fortified town on a rise of ground and built the present town the following year, some 6 kilometers from the original site, on the banks of the Canche.