Text Generation
Transformers
TensorBoard
Safetensors
Japanese
English
mistral
conversational
text-generation-inference
leonardlin commited on
Commit
63f3d39
·
verified ·
1 Parent(s): d1dc656

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -97,7 +97,7 @@ We believe these benchmarks will be generally useful and plan to open-source the
97
 
98
  All Shisa V2 models inherit the [chat templates](https://huggingface.co/docs/transformers/v4.37.1/chat_templating) of their respective base models and have been tested and validated for proper inference with both [vLLM](https://github.com/vllm-project/vllm) and [SGLang](https://github.com/sgl-project/sglang).
99
 
100
- We recommend running at lower temperatures: 0.0-0.2 for factual answers and 0.5-0.7 for creative tasks.
101
 
102
  No additional safety alignment has been done on these models, so they will largely inherit the base models' biases and safety profiles.
103
 
 
97
 
98
  All Shisa V2 models inherit the [chat templates](https://huggingface.co/docs/transformers/v4.37.1/chat_templating) of their respective base models and have been tested and validated for proper inference with both [vLLM](https://github.com/vllm-project/vllm) and [SGLang](https://github.com/sgl-project/sglang).
99
 
100
+ Running sampler sweeps, we found the models operate well across a variety of temperatures in most settings. For translation tasks specifically, we recommend a lower temperatures (0.2) to increase accuracy. For role-play and creative tasks, a higher temp (eg 1.0) seems to give good results. To prevent cross-lingual token leakage we recommend a top_p of 0.9 or min_p of 0.1.
101
 
102
  No additional safety alignment has been done on these models, so they will largely inherit the base models' biases and safety profiles.
103