Update README.md
Browse files
README.md
CHANGED
|
@@ -5,6 +5,7 @@ Based on DictaLM2.0; fine-tuned for text summarization.
|
|
| 5 |
Known Issues:
|
| 6 |
- The model is bloated (disk size).
|
| 7 |
- While the results look pretty good, the model was not evaluated.
|
|
|
|
| 8 |
|
| 9 |
|
| 10 |
# Data:
|
|
@@ -56,7 +57,11 @@ model = AutoModelForCausalLM.from_pretrained(
|
|
| 56 |
model.to('cuda')
|
| 57 |
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
|
| 58 |
|
| 59 |
-
text = '
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
summarize(text, max_new_tokens=512, tokenizer=tokenizer, model=model)
|
| 62 |
```
|
|
|
|
| 5 |
Known Issues:
|
| 6 |
- The model is bloated (disk size).
|
| 7 |
- While the results look pretty good, the model was not evaluated.
|
| 8 |
+
- Short inputs (i.e., "articles" of one line) will yield a contextless "summary".
|
| 9 |
|
| 10 |
|
| 11 |
# Data:
|
|
|
|
| 57 |
model.to('cuda')
|
| 58 |
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
|
| 59 |
|
| 60 |
+
text = '''
|
| 61 |
+
诇驻注诪讬诐, 诪转讞砖拽 诇讞讝讜专 讗讞讜专讛 讘讝诪谉. 讗转诐 讬讜讚注讬诐, 诇讬诪讬诐 讛讟讜讘讬诐 讛讛诐. 诇讬诪讬诐 砖诇 讚讬讬讘 讜砖诇 住讜讙专 砖讟讞讬诐, 砖诇 拽讬谉 讜砖诇 驻拽诪谉, 砖诇 讗诇讗讚讬谉 讜砖诇 诪诇讱 讛讗专讬讜转, 砖诇 讛讜讙讜 讜砖诇 讜讜专诪住, 砖诇 讟讟专讬住 讜砖诇 讻诇 砖讗专 讛拽诇讗住讬拽讜转 砖讗讛讘谞讜 讻诇 讻讱...
|
| 62 |
+
|
| 63 |
+
讻讗谉, 讘"诪住注 讗诇 讛注讘专", 讗住驻谞讜 讗转 讻诇 讗讜转诐 诪砖讞拽讬诐 讬砖谞讬诐, 讜讗谞讜 诪爪讬注讬诐 诇讻诐 讗讜转诐, 讬讞讚 注诐 转讗讜专讬诐, 转诪讜谞讜转, 拽讟讙讜专讬讜转, 爪'讬讟讬诐, 驻转专讜谞讜转 讜注讜讚 - 讻讚讬 砖讙诐 讗转诐 转讜讻诇讜 诇讞讝讜专 讞讝专讛 讘讝诪谉 - 讜诇讛谞讜转 诪讛谞讜住讟诇讙讬讛.
|
| 64 |
+
'''.strip()
|
| 65 |
|
| 66 |
summarize(text, max_new_tokens=512, tokenizer=tokenizer, model=model)
|
| 67 |
```
|