Secret passages do exist they were for the safety of the royals to get them out of the castle safely during an attack. Visitors would never be privy to them. Only trusted members of the house hold would know where they are.