DRY parameters?
Could you be a bit clearer about the DRY parameters?
Is the "0.8 1.75 4" supposed to be these, or something else? llama.cpp also allows a --dry-penalty-last-n
- DRY multiplier: 0.8
- DRY allowed length: 1.75
- DRY base: 4.0
Sure,
Dry multiplier: 0.8
Dry base: 1.75
Dry allowed length: 4
This dry setting is pretty light, as it will only penalize chunks of text where there's four or more repeated tokens.
Dry penalty last-n I assume is referring to range, which can be left blank. That value lets you specify how far back through your context it should look for repetitions. By default I believe it looks over the entire context for repetitions but if you specify a context range, it will only penalize repetitions within that context. To increase the potency of dry, I'd recommend increasing the multiplier.
If struggling with repetition, I'd also recommend adding in repetition penalty between 1.05 - 1.1 and messing around with the temp / minp & topp samplers or even XTC / smooth sampling.