fireballoon
commited on
Commit
·
e1c582c
1
Parent(s):
f7bff14
Update README.md
Browse files
README.md
CHANGED
@@ -25,11 +25,30 @@ tokenizer = AutoTokenizer.from_pretrained("fireballoon/baichuan-vicuna-7b", use_
|
|
25 |
model = AutoModelForCausalLM.from_pretrained("fireballoon/baichuan-vicuna-7b")
|
26 |
```
|
27 |
|
28 |
-
# Inference
|
|
|
29 |
```
|
30 |
python3 -m fastchat.serve.cli --model-path fireballoon/baichuan-vicuna-7b
|
31 |
```
|
32 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
33 |
# Test examples on FastChat Eval
|
34 |
Test example on [FastChat Eval](https://github.com/lm-sys/FastChat/blob/main/fastchat/eval/table/question.jsonl)
|
35 |
```
|
|
|
25 |
model = AutoModelForCausalLM.from_pretrained("fireballoon/baichuan-vicuna-7b")
|
26 |
```
|
27 |
|
28 |
+
# Inference
|
29 |
+
Inference with FastChat:
|
30 |
```
|
31 |
python3 -m fastchat.serve.cli --model-path fireballoon/baichuan-vicuna-7b
|
32 |
```
|
33 |
|
34 |
+
Inference with Transformers:
|
35 |
+
```ipython
|
36 |
+
>>> from transformers import AutoTokenizer, AutoModelForCausalLM, TextStreamer
|
37 |
+
>>> tokenizer = AutoTokenizer.from_pretrained("fireballoon/baichuan-vicuna-7b", use_fast=False)
|
38 |
+
>>> model = AutoModelForCausalLM.from_pretrained("fireballoon/baichuan-vicuna-7b").half().cuda()
|
39 |
+
>>> instruction = "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: {} ASSISTANT:"
|
40 |
+
>>> prompt = instruction.format("five tips to help with sleep") # user message
|
41 |
+
>>> generate_ids = model.generate(tokenizer(prompt, return_tensors='pt').input_ids.cuda(), max_new_tokens=2048, streamer=streamer)
|
42 |
+
'''
|
43 |
+
1. Create a relaxing bedtime routine, such as winding down with a warm bath or reading a book before bed.
|
44 |
+
2. Avoid caffeine and alcohol close to bedtime, as they can disrupt sleep.
|
45 |
+
3. Use blackout curtains or shades to block out light from the outside, which can interfere with your body's natural sleep rhythms.
|
46 |
+
4. Keep your bedroom cool and dark, and avoid using electronic devices at least an hour before bed.
|
47 |
+
5. Regular exercise and a healthy diet can also help improve sleep quality.
|
48 |
+
'''
|
49 |
+
```
|
50 |
+
|
51 |
+
|
52 |
# Test examples on FastChat Eval
|
53 |
Test example on [FastChat Eval](https://github.com/lm-sys/FastChat/blob/main/fastchat/eval/table/question.jsonl)
|
54 |
```
|