|
--- |
|
library_name: transformers |
|
tags: [] |
|
--- |
|
|
|
## Llama-3-SURPASSONE-JP-8B |
|
|
|
 |
|
|
|
### Model Description |
|
|
|
**Llama-3-SURPASSONE-JP-8B** is a large language model trained by [SURPASSONE, Inc](https://surpassone.com/). |
|
Based on [meta-llama/Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct), this model has undergone additional post-training of Japanese to expand instruction-following capabilities in Japanese. |
|
|
|
This model is specialized in generating MCQ questions with options, correct answers and corresponding explanations on “Nursing Care” if given a specific topic. Also, it can generate answers based on questions on “Nursing Care”. |
|
|
|
For more details, please refer to [our blog post](https://docs.google.com/document/d/1ENAEzgV3n-sFiSoV3oQBTgzjeyTfmL64zEczepTKEW0/edit?usp=sharing). |
|
|
|
### Usage |
|
|
|
```python |
|
# make sure you are logged in huggingface |
|
hf_token = "" # your huggingface token |
|
from huggingface_hub import login |
|
login() |
|
|
|
import torch |
|
from transformers import AutoModelForCausalLM, BitsAndBytesConfig, AutoTokenizer |
|
|
|
bnb_config = BitsAndBytesConfig( |
|
load_in_4bit=True, |
|
bnb_4bit_use_double_quant=True, |
|
bnb_4bit_quant_type="nf4", |
|
bnb_4bit_compute_dtype=torch.bfloat16, |
|
) |
|
|
|
model_id = "surpassone/Llama-3-SURPASSONE-JP-8B" |
|
|
|
tokenizer = AutoTokenizer.from_pretrained(model_id) |
|
model = AutoModelForCausalLM.from_pretrained( |
|
model_id, |
|
device_map="auto", |
|
quantization_config=None, # Use bnb_config, if need to use 4 bit quantization else None |
|
) |
|
if tokenizer.pad_token is None: |
|
tokenizer.pad_token = tokenizer.eos_token |
|
|
|
model.eval() |
|
|
|
# for MCQ set generation |
|
|
|
alpaca_prompt = """以下は、タスクを説明する指示と、さらに詳しいコンテキストを提供する入力を組み合わせたものです。要求を適切に完了する応答を記述してください。 |
|
|
|
### 説明書: |
|
{} |
|
|
|
### 入力: |
|
{} |
|
|
|
### 応答: |
|
{}""" |
|
|
|
EOS_TOKEN = "<|endoftext|>" # Define the EOS token, adjust according to your tokenizer |
|
|
|
inputs = tokenizer( |
|
[ |
|
alpaca_prompt.format( |
|
"次のトピックに関する複数選択問題を生成します。", # instruction |
|
"介護：体の仕組み", # input |
|
"", # output - leave this blank for generation! |
|
) |
|
], return_tensors = "pt").to("cuda") |
|
|
|
from transformers import TextStreamer |
|
text_streamer = TextStreamer(tokenizer) |
|
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 1028) |
|
|
|
#for QA generation |
|
|
|
# Define the formatting function and the prompt template |
|
alpaca_prompt = """以下は質問です。質問に適切に答える回答を書いてください。 |
|
|
|
### 質問： |
|
{} |
|
|
|
### 答え： |
|
{}""" |
|
|
|
eos_token_id = tokenizer.eos_token_id |
|
|
|
inputs = tokenizer( |
|
[alpaca_prompt.format( |
|
"介護福祉士はどのような責任を負うべきですか？", # Question |
|
"" # Answer - leave this blank for generation! |
|
)], |
|
return_tensors="pt" |
|
).to("cuda") |
|
|
|
from transformers import TextStreamer |
|
text_streamer = TextStreamer(tokenizer) |
|
_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 1028) |
|
``` |
|
|
|
### Developers |
|
|
|
Listed in alphabetical order. |
|
|
|
- [Leo Uno](https://huggingface.co/leouno12) |
|
- [Mustain Billah](https://huggingface.co/Mustain) |
|
- [Shugo Saito](https://huggingface.co/shugo3110) |
|
|
|
|
|
### License |
|
|
|
[Meta Llama 3 Community License](https://llama.meta.com/llama3/license/) |
|
|
|
### How to Cite |
|
|
|
```tex |
|
@misc{surpassonellama2024, |
|
title={surpassone/Llama-3-SURPASSONE-JP-8B}, |
|
url={https://huggingface.co/surpassone/Llama-3-SURPASSONE-JP-8B}, |
|
author={Mustain Billah and Shugo Saito and Leo Uno}, |
|
year={2024}, |
|
} |
|
``` |
|
|
|
### Citations |
|
|
|
```tex |
|
@article{llama3modelcard, |
|
title={Llama 3 Model Card}, |
|
author={AI@Meta}, |
|
year={2024}, |
|
url = {https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md} |
|
} |
|
``` |
|
|
