I just now figured out how to properly respond to you (I'm still a bit of a noob) so my reply is up a bit, but I'll rewrite it here.

He didn't expressly tell me not to go through his phone while he napped, but he didn't give me the go ahead, either, which is why I'm uncomfortable with it.

The nervousness about angering him is a PTSD thing from a past relationship, though I am working on that.