I don’t see a relu between embedding and GRU. Maybe you can point me to the exact line you are looking at?
Choosing among different non-linearities is an interesting topic. Relu is a popular choice because it often gives good results in practice.