The castle, together with the Crichton lands, passed to James II of Scotland in 1453, and the castle has been crown property ever since.[2] It served as a state prison, holding such prisoners as Cardinal Beaton and the 6th Earl of Angus.[3]