Tracing lets you use dynamic behavior in Python since it just records tensor operations. This may work better for your use case, but as you pointed out there are some fundamental limitations such as the inability to trace control flow / Python values. They both should be pretty easy to try out on your codebase (i.e. call trace() or script() on a module). If tracing does not work out of the box it is unlikely that you can maintain the semantics of your model and get it working under tracing (i.e. if you use control flow). Scripting may require some work to get your model using only features supported by the compiler 122, but you will probably be able to get it working with some code changes.
GRU and LSTM can both be compiled on master/nightly, (AFAIK only LSTM is in the v1.2.0 release), we haven’t had many requests for RNN yet. If you’d like to see it (or something else in PyTorch) be script-able that isn’t already 16, please file an issue on GitHub.