Why is your GRU layer not part of your network? JWP contains only four linear layers, so the GRU layer doesn’t get train at all. You might even want to make the embeddings as part of the network, even if you don’t want to train them.