Nah, you did know why he was a scary mofo to everyone else. Tall and imposing to them, clearly unhinged, hates being questioned, waves around an unpowered magic staff, the whole shiteree. It’s just that he’s never been scary for you.