RoPE
[Interpolation](https://texonom.com/interpolation-bce3644c1d424ebda3d300da73972609) 


### ROTARY POSITION EMBEDDING




[llama : add custom RoPE (#2054)](https://github.com/ggerganov/llama.cpp/commit/6e7cca404748dd4b1a3affd0d1296e37f4ac0a6f)
> [Untitled](https://arxiv.org/pdf/2306.15595.pdf)

> [NTK-Aware Scaled RoPE allows LLaMA models to have extended (8k+) context size without any fine-tuning and minimal perplexity degradation.](https://www.reddit.com/r/LocalLLaMA/comments/14lz7j5/ntkaware_scaled_rope_allows_llama_models_to_have/)





