Thanks for your reply! I try to use a standard transformer model and I found a strange error like this:

I/INFO token: 3118
I/INFO input tensor: [3118]
I/INFO input tensor shape: [1, 1, 1]
I/INFO input length tensor: [1]
I/INFO input length tensor shape: [1]
I/INFO input tensor test: [3118]
E/AndroidRuntime: FATAL EXCEPTION: Thread-2
Process: com.example.seq2seqnmt, PID: 12138
com.facebook.jni.CppException: shape ‘[2, -1, 8, 32]’ is invalid for input of size 256
debug_handle:-1
Exception raised from run at ../torch/csrc/jit/mobile/module.cpp:206 (most recent call first):
(no backtrace available)
    at org.pytorch.LiteNativePeer.forward(Native Method)
    at org.pytorch.Module.forward(Module.java:52)
    at org.pytorch.demo.seq2seqnmt.MainActivity.translate(MainActivity.java:237)
    at org.pytorch.demo.seq2seqnmt.MainActivity.run(MainActivity.java:108)
    at java.lang.Thread.run(Thread.java:764)

D/EGL_emulation: eglMakeCurrent: 0xa8a850c0: ver 3 0 (tinfo 0xa8a83210)
E/SpellCheckerSession: ignoring processOrEnqueueTask due to unexpected mState=TASK_CLOSE scp.mWhat=TASK_CLOSE

When I transforme the pytorch model to lite model, my input batch size is 2. And the input batch size now is 1 as the log shown. But it seems the batch size which is in the first dimension is still 2, that’s a little weired.
I know that tf-lite model will keep the length or batch size all the same when transforming its python model to lite or cpp model. So is pytorch do the same things like fix the sequence length and batch size. If so, should I do a resize on the model at the time of inference, as far as I know tf-lite will have this handled.