At this time Edinburgh was still a very modest town , but David 's successor , Malcolm IV ( 1153 1165 ) , made its castle his main residence .
David 's successor was Queen Margaret the III and she decided to move to Edinburgh castle .