Context size 4096 tokens?
#2
by
Pentium95
- opened
Does anyone have tested this quant with higher context size? Does this work fine with YaRN? Which context sizes are achievable?
With Llama2s you can use rope ; see the very bottom of this page for rope info:
https://huggingface.co/DavidAU/TieFighter-Holodeck-Holomax-Mythomax-F1-V1-COMPOS-20B-gguf
NOTE:
This might be outdated some what, as this is 6 months old.