The Vieuxponts were a powerful land-owning family in North West England, who also owned the castles of Appleby and Brough. In 1264, Robert de Vieuxpont's grandson, also named Robert, was declared a traitor, and his property was confiscated by Henry III. Brougham Castle and the other estates were eventually returned to the Vieuxpont family, and stayed in their possession, until 1269, when the estates passed to the Clifford family through marriage.