The optimization line slows down my model:
torchscript_model_optimized = optimize_for_mobile(torchscript_model)

The model contains a CNN and a bidirectional LSTM for 600 steps. On a pixel 4 it gets slowed down from 5s to 16s when I activate the optimization.