Solved by sarnikowski in post #2 
              
              
                The solution was to make the batch size dynamic as well. This required me to input the initial hidden state to the forward pass as well, when tracing with ONNX.