The castle, which was sited on a rocky promontory, was founded by the Normans during their invasion of West Wales in the 12th century. A stone tower was built on the headland's highest point which was protected by a curtain wall. The walls had a gateway and several small towers on the landward side.