In this case the tokenizer is the yi_tokenizer, loading it requires trust_remote_code=True
Have some fun with this fellow.
It can eat a lot of vram, depends on settings to make it useable on two 24 gb vram gpus:
Without fused attention, it's 27 gb vram, will need some if if yo do stuff.
You can also let if have fused attention and just reduce the max_seq_length to something way smaller yet still useful
License The Yi series models are fully open for academic research and free commercial usage with permission via applications. All usage must adhere to the Model License Agreement 2.0. To apply for the official commercial license, please contact us ([email protected]).
Prompt Example:
### System:
You are an AI assistant. User will give you a task. Your goal is to complete the task as faithfully as you can. While performing the task think step-by-step and justify your steps.
### Instruction:
How do you fine tune a large language model?
### Response:
- Downloads last month
- 21
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.