File size: 2,935 Bytes
faabd97 c34e078 faabd97 c34e078 faabd97 569d15c faabd97 c34e078 faabd97 c34e078 faabd97 3898c0b faabd97 cfdb1f8 3898c0b faabd97 3898c0b faabd97 3898c0b 569d15c 3898c0b faabd97 c34e078 faabd97 c34e078 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 |
---
library_name: transformers
tags:
- sentiment
- classifier
license: mit
datasets:
- financial_phrasebank
language:
- en
---
### Model Description
<!-- Provide a longer summary of what this model is. -->
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
- **Developed by:** Mit Patel
- **Model type:** Text generation/ classifier
- **Language(s) (NLP):** English
- **Finetuned from model :** Phi-2
## Training Details
https://github.com/mit1280/fined-tuning/blob/main/phi_2_classification_fine_tune.ipynb
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- training_steps: 10000
### Inference
```python
!pip install -q transformers==4.37.2 accelerate==0.27.0
import re
from transformers import AutoTokenizer, AutoModelForCausalLM, StoppingCriteria
import torch
tokenizer = AutoTokenizer.from_pretrained("Mit1208/phi-2-classification-sentiment-merged")
model = AutoModelForCausalLM.from_pretrained("Mit1208/phi-2-classification-sentiment-merged", device_map="auto", trust_remote_code=True).eval()
class EosListStoppingCriteria(StoppingCriteria):
def __init__(self, eos_sequence = tokenizer.encode("<|im_end|>")):
self.eos_sequence = eos_sequence
def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
last_ids = input_ids[:,-len(self.eos_sequence):].tolist()
return self.eos_sequence in last_ids
inf_conv = [{'from': 'human',
'value': "Text: In sales volume , Coca-Cola 's market share has decreased by 2.2 % to 24.2 % ."},
{'from': 'phi', 'value': "I've read this text."},
{'from': 'human',
'value': 'Please determine the sentiment of the given text and choose from the options: Positive, Negative, Neutral, or Cannot be determined.'}]
# need to load because model doesn't has classifer head.
id2label = {0: 'negative', 1: 'neutral', 2: 'positive'}
inference_text = tokenizer.apply_chat_template(inf_conv, tokenize=False) + '<|im_start|>phi:\n'
inputs = tokenizer(inference_text, return_tensors="pt", return_attention_mask=False).to('cuda')
outputs = model.generate(inputs["input_ids"], max_new_tokens=1024, pad_token_id= tokenizer.eos_token_id,
stopping_criteria = [EosListStoppingCriteria()])
text = tokenizer.batch_decode(outputs)[0]
answer = text.split("<|im_start|>phi:")[-1].replace("<|im_end|>", "").replace(".", "")
sentiment_label = re.search(r'(\d)', answer)
sentiment_score = int(sentiment_label.group(1))
if sentiment_score:
print(id2label.get(sentiment_score, "none"))
else:
print("none")
```
### Framework versions
- PEFT 0.8.2
- Transformers 4.37.2
- Pytorch 2.1.0+cu121
- Datasets 2.16.1
- Tokenizers 0.15.1 |