Solved by isaacmg in post #3 
              
              
                Yeah I was able to solve it by applying StandardScaler on my the data and filling all NaN values. Large values even more so than with traditional RNNs seem to cause loss to explode.