Update README.md
Browse files
README.md
CHANGED
@@ -62,4 +62,14 @@ pipeline = transformers.pipeline(
|
|
62 |
|
63 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
64 |
print(outputs[0]["generated_text"])
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
65 |
```
|
|
|
62 |
|
63 |
outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
|
64 |
print(outputs[0]["generated_text"])
|
65 |
+
```
|
66 |
+
|
67 |
+
```
|
68 |
+
Loading checkpoint shards: 100%|βββββββββββββββββββββββββββββββββββββββββββββββββββ| 2/2 [00:07<00:00, 3.75s/it]
|
69 |
+
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
|
70 |
+
Setting `pad_token_id` to `eos_token_id`:128001 for open-end generation.
|
71 |
+
<|im_start|>user
|
72 |
+
What is a large language model?<|im_end|>
|
73 |
+
<|im_start|>assistant
|
74 |
+
A large language model (LLM) is a deep neural network that is trained to predict the next word in a sequence of text. LLMs are typically trained on large amounts of text data and can be used for a variety of tasks such as language translation, text completion, and question answering. They are often used to generate human-like text and are becoming increasingly popular in natural language processing applications. The LLM uses a transformer architecture, which consists of multiple layers of neural networks that are trained to process and understand the relationships between words in a sentence. The transformer architecture is designed to handle long sequences of text and is capable of capturing the context of a word within a sentence. This allows the LLM to generate coherent and grammatically correct text that is similar to human writing. LLMs are typically trained on a large corpus of text data and can be fine-tuned for specific tasks by retraining on smaller datasets that are relevant to the task at hand. This allows the LLM to adapt to the specific requirements of a particular application and improve its performance. The LLM can be used to generate text in a variety of formats, including natural language, code, and even mathematical expressions. It can also be used to translate text from one language to another, generate summaries of
|
75 |
```
|