Why LSTM instead of GRU? Why the output and not the hidden state of an RNN? Why ReLu and not Tanh, Sigmoid, etc.? Why not more linear layers before the output? …? 
There’s not one-size-fits-all solution, even for basic network models and/or for the same task (e.g., sentence classification). At the end of the day, there’s a lot of trial-and-error involved when it comes the training. Sure, it’s a good approach to start with tried-and-tested approaches. For example 50, I use the hidden state of an RNN to train a text classifier.