In 1470 Mehmet the Conqueror further strengthened the ancient portal by building three towers of his own , linked by curtain walls to the four Byzantine towers flanking the Golden Gate .
The portal was removed and replaced with three towers .