Maybe, if he was hopeful, he was allowed to exit the castle whenever he wanted? But, it was quickly squashed because he would still be surrounded by guards. That had to be the only reason he was allowed to leave, with the expectation of returning later.