I have the same question.
Intuitively, if we normalize over individual episodes, then episodes with a high overall reward will be normalized into the same range as episodes with very poor rewards.
For example, consider we take actions at states that give us a near constant -50 reward for all time steps, and another other where we receive +50. Then, all actions in both episodes will be reinforced to the same degree after we z-score them. This will still properly reinforce on a per action basis within episodes, but effectively reduce all episodes to having the same amount of positive and negative reinforcement. I can only see this per action normalization working in a true MDP environment.