Update README.md

README.md

@@ -29,22 +29,10 @@ This is the model card of a 🤗 transformers model that has been pushed on the
 - **Language(s) (NLP):** English
 - **Finetuned from model :** Phi-2
 
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
 
 ## Training Details
 
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure 
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
 
 
@@ -69,12 +57,57 @@ The following hyperparameters were used during training:
 
 <!-- This section describes the evaluation protocols and provides the results. -->
 
-### Testing Data, Factors & Metrics
+### Inference
 
+```
+!pip install -q transformers==4.37.2  accelerate==0.27.0
 
-### Results
+import re
+from transformers import AutoTokenizer, AutoModelForCausalLM, StoppingCriteria
+import torch
 
-[More Information Needed]
+tokenizer = AutoTokenizer.from_pretrained("Mit1208/phi-2-classification-sentiment-merged")
+model = AutoModelForCausalLM.from_pretrained("Mit1208/phi-2-classification-sentiment-merged", device_map="auto", trust_remote_code=True).eval()
+
+class EosListStoppingCriteria(StoppingCriteria):
+    def __init__(self, eos_sequence = tokenizer.encode("<|im_end|>")):
+        self.eos_sequence = eos_sequence
+
+    def __call__(self, input_ids: torch.LongTensor, scores: torch.FloatTensor, **kwargs) -> bool:
+        last_ids = input_ids[:,-len(self.eos_sequence):].tolist()
+        return self.eos_sequence in last_ids
+
+inf_conv = [{'from': 'human',
+  'value': "Text: In sales volume , Coca-Cola 's market share has decreased by 2.2 % to 24.2 % ."},
+ {'from': 'phi', 'value': "I've read this text."},
+ {'from': 'human',
+  'value': 'Please determine the sentiment of the given text and choose from the options: Positive, Negative, Neutral, or Cannot be determined.'}]
+# need to load because model doesn't has classifer head.
+
+id2label = {0: 'negative', 1: 'neutral', 2: 'positive'}
+
+inference_text = tokenizer.apply_chat_template(inf_conv, tokenize=False) + '<|im_start|>phi:\n'
+inputs = tokenizer(inference_text, return_tensors="pt", return_attention_mask=False).to('cuda')
+outputs = model.generate(inputs["input_ids"], max_new_tokens=1024, pad_token_id= tokenizer.eos_token_id,
+            stopping_criteria = [EosListStoppingCriteria()])
+
+text = tokenizer.batch_decode(outputs)[0]
+
+# print(text.split("The correct option is")[-1].replace("<|im_end|>", "").replace(".", ""))
+
+# Define a dictionary to map values to labels
+label_map = {"2": "positive", "0": "negative", "1": "neutral"}
+answer = text.split("<|im_start|>phi:")[-1].replace("<|im_end|>", "").replace(".", "")
+
+sentiment_label = re.search(r'(\d)', answer)
+sentiment_score = int(sentiment_label.group(1))
+
+if sentiment_label:
+    sentiment_score = int(sentiment_label.group(1))
+    print(id2label.get(sentiment_score, "none"))
+else:
+    print("none")
+```
 
 #### Summary