Exporting the operator repeat_interleave to ONNX opset version (<=12) is not supported!
When I export Roformer model to onnxruntime, this error occurred! Is there any ops can replace 'torch.repeat_interleave'?
It's in [https://github.com/huggingface/transformers/blob/master/src/transformers/models/roformer/modeling_roformer.py](url), at line 330 and 332
I have a solution, but maybe it's not the best way.

sin_pos = torch.zeros_like(sin).repeat(1, 1, 1, 2)
sin_pos[..., ::2] = sin
sin_pos[..., 1::2] = sin
cos_pos = torch.zeros_like(cos).repeat(1, 1, 1, 2)
cos_pos[..., ::2] = cos
cos_pos[..., 1::2] = cos<|||||>It seems that you haven't change roformer's positional embedding codes yet. Demo codes are like:
```python
import torch
sinusoidal_pos = torch.randn(1,12,16,32)
sin, cos = sinusoidal_pos.chunk(2, dim=-1)

sin_pos = torch.repeat_interleave(sin, 2, dim=-1)
cos_pos = torch.repeat_interleave(cos, 2, dim=-1)

sin_pos_newway = torch.stack([sin,sin],axis=-1).reshape_as(sinusoidal_pos)
cos_pos_newway = torch.stack([cos,cos],axis=-1).reshape_as(sinusoidal_pos)

assert sin_pos.equal(sin_pos_newway)
assert cos_pos.equal(cos_pos_newway)
```