Indeed, REINFORCE is not that great in order to learn features through linear layers. Adding a prediction of values increases the speed to learn relevant features in the hidden layer. That’s why actor-critic is much more stable, and still work with 30 neurons.