Push model using huggingface_hub.
Browse files- README.md +3 -74
- config.json +1 -1
- model.safetensors +2 -2
README.md
CHANGED
@@ -1,81 +1,10 @@
|
|
1 |
---
|
|
|
2 |
tags:
|
3 |
- model_hub_mixin
|
4 |
- pytorch_model_hub_mixin
|
5 |
---
|
6 |
|
7 |
This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
|
8 |
-
- Library:
|
9 |
-
- Docs: [More Information Needed]
|
10 |
-
|
11 |
-
---
|
12 |
-
|
13 |
-
# GPT-2 Nepali Model
|
14 |
-
|
15 |
-
This repository contains a custom GPT-2 model trained on Nepali text. Follow the instructions below to use this model for text generation.
|
16 |
-
|
17 |
-
---
|
18 |
-
|
19 |
-
## How to Use the Model
|
20 |
-
|
21 |
-
1. **Download the Required Code**
|
22 |
-
Save the [`model_code.py`](https://github.com/Aananda-giri/llm.np/blob/main/3.%20GPT-2/sebastian_gutenberg/huggingface_hub/model_code.py) file in the same directory where you'll run the script.
|
23 |
-
|
24 |
-
2. **Install Required Libraries**
|
25 |
-
Ensure you have the necessary libraries installed:
|
26 |
-
```bash
|
27 |
-
pip install transformers torch
|
28 |
-
```
|
29 |
-
|
30 |
-
3. **Run the Following Code**
|
31 |
-
Here's an example to load the model and generate text:
|
32 |
-
|
33 |
-
```python
|
34 |
-
import torch
|
35 |
-
from model_code import GPTModel, generate_and_print_sample
|
36 |
-
from transformers import PreTrainedTokenizerFast
|
37 |
-
|
38 |
-
# Load the tokenizer
|
39 |
-
tokenizer = PreTrainedTokenizerFast.from_pretrained("Aananda-giri/NepaliBPE")
|
40 |
-
|
41 |
-
# Define the starting text
|
42 |
-
start_context = "रामले भात"
|
43 |
-
|
44 |
-
# Load the pre-trained model
|
45 |
-
loaded_model = GPTModel.from_pretrained("Aananda-giri/GPT2-Nepali")
|
46 |
-
|
47 |
-
# Move the model to the appropriate device (CPU or GPU)
|
48 |
-
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
49 |
-
loaded_model.to(device)
|
50 |
-
|
51 |
-
# Generate text
|
52 |
-
generate_and_print_sample(
|
53 |
-
loaded_model, tokenizer, device, start_context
|
54 |
-
)
|
55 |
-
```
|
56 |
-
|
57 |
-
---
|
58 |
-
|
59 |
-
## Additional Notes
|
60 |
-
|
61 |
-
- **Tokenizer**: The model uses a pre-trained tokenizer available at `Aananda-giri/NepaliBPE`. Ensure this is downloaded and accessible during runtime.
|
62 |
-
- **Dependencies**: This code requires `transformers` (by Hugging Face) and `torch` (PyTorch). Install them if not already installed.
|
63 |
-
- **Device Compatibility**: The script automatically detects if a CUDA-enabled GPU is available and utilizes it for faster inference. If not, it defaults to the CPU.
|
64 |
-
|
65 |
-
---
|
66 |
-
|
67 |
-
## Example Output
|
68 |
-
|
69 |
-
Input:
|
70 |
-
```
|
71 |
-
रामले भात
|
72 |
-
```
|
73 |
-
|
74 |
-
Generated Text:
|
75 |
-
```
|
76 |
-
रामले भात खाएर सन्तोष माने। ऊ आफ्ना साथीहरूसँग रमाइलो गरिरहेको थियो।
|
77 |
-
```
|
78 |
-
|
79 |
-
---
|
80 |
-
|
81 |
-
Let me know if you'd like further assistance!
|
|
|
1 |
---
|
2 |
+
pipeline_tag: text-generation
|
3 |
tags:
|
4 |
- model_hub_mixin
|
5 |
- pytorch_model_hub_mixin
|
6 |
---
|
7 |
|
8 |
This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
|
9 |
+
- Library: https://huggingface.co/Aananda-giri/GPT2-Nepali/
|
10 |
+
- Docs: [More Information Needed]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
config.json
CHANGED
@@ -1,6 +1,6 @@
|
|
1 |
{
|
2 |
"cfg": {
|
3 |
-
"context_length":
|
4 |
"drop_rate": 0.1,
|
5 |
"emb_dim": 768,
|
6 |
"n_heads": 12,
|
|
|
1 |
{
|
2 |
"cfg": {
|
3 |
+
"context_length": 512,
|
4 |
"drop_rate": 0.1,
|
5 |
"emb_dim": 768,
|
6 |
"n_heads": 12,
|
model.safetensors
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
-
size
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:c3e150c63e41f5bd75bec802c5c1671e1ea94a688dbc3638ad0358afc7d4657a
|
3 |
+
size 661486448
|