Extremely sensitive to parameters
#3
by
ilintar
- opened
I'm testing a Q5_K_S, so one would think it would behave well. However, the model is extremely sensitive to generation parameters and with the recommended ones, it actually devolves into a loop where it starts outputting less and less coherent thinking messages and finally just spouts nonsense words.
The only parameter combination I've actually succeeded with to get coherent output out of the models looks like this:
- temperature: 0.5
- top_k: 15 (!)
- repeat_penalty: 1.1
- min_p: 0
- top_p: 0.9