Update README.md

README.md

@@ -8,14 +8,35 @@ metrics:
 - bleu
 model-index:
 - name: neutral_job_title_rephraser_pl
-  results: []
+  results:
+  - task:
+      type: text2text-generation
+      name: Gender-Neutral Job Title Rephrasing
+    dataset:
+      type: ArielUW/jobtitles
+      name: Job Titles Dataset
+      config: default
+      split: test
+    metrics:
+    - type: bleu
+      value: 93.9441
+      name: BLEU
+    - type: precision  
+      value: 1.0
+      name: Attempted Noun Neutralisation Precision
+    - type: recall
+      value: 0.892
+      name: Attempted Noun Neutralisation Recall
+    - type: levenshtein  
+      value: 0.0395
+      name: Normalized Levenshtein Distance (neutralization needed)
+    - type: levenshtein  
+      value: 0.0001
+      name: Normalized Levenshtein Distance (neutralization not needed)
 datasets:
 - ArielUW/jobtitles
 ---
 
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-
 # model_output
 
 This model is a fine-tuned version of [facebook/m2m100_1.2B](https://huggingface.co/facebook/m2m100_1.2B) on [ArielUW/jobtitles](https://huggingface.co/datasets/ArielUW/jobtitles) dataset.
@@ -37,12 +58,55 @@ Sentences not containing such terms are not expected to change at all, for examp
 
 In terms of actual outcomes and errors in outputs, see our [readme](https://github.com/ArielUW/IMLLA-FinalProject/blob/main/README.md).
 
+# Model usage
+
+To use this model, you will need to install the transformers and sentencepiece libraries:
+
+  !pip install transformers sentencepiece
+
+You can then use the model directly through the pipeline API, which provides a high-level interface for text generation:
+  from transformers import pipeline
+  pipe = pipeline("text2text-generation", model="mongrz/model_output")
+  gender_neutral_text = pipe("Pielęgniarki protestują pod sejmem.")
+  print(gender_neutral_text)
+  #expected output: [{'generated_text': 'Osoby pielęgniarskie protestują pod sejmem.'}] 
+This will create a pipeline object for text-to-text generation using your model. You can then pass the input text to the pipe object to generate the gender-neutral version. The output will be a list of dictionaries, each containing the generated text.
+Alternatively, you can still load the tokenizer and model manually for more fine-grained control:
+  from transformers import AutoModelForSeq2SeqLM, AutoTokenizer
+  
+  tokenizer = AutoTokenizer.from_pretrained("mongrz/model_output")
+  model = AutoModelForSeq2SeqLM.from_pretrained("mongrz/model_output")
+  
+  text_to_translate = "Pielęgniarki protestują pod sejmem." 
+  model_inputs = tokenizer(text_to_translate, return_tensors="pt")
+  
+  #Generate gender-neutral text
+  gen_tokens = model.generate(**model_inputs, forced_bos_token_id=tokenizer.get_lang_id("pl"))
+  
+  #Decode and print the generated text 
+  print(tokenizer.batch_decode(gen_tokens, skip_special_tokens=True))
+This approach allows you to access the tokenizer and model directly and customize the generation process further if needed. Choose the method that best suits your needs.
+  
 ## Intended uses & limitations
-The model has only been fine-tuned for single-sentence inputs, so other types of inputs can be unstable. So far, it underperforms for low-frequency items, morphosyntactically complex cases and feminine nouns. 
+While this model demonstrates promising results in generating gender-neutral job titles in Polish, it has certain limitations:
+
+    Low-Frequency Items: The model may struggle with less common job titles or words that were not frequently present in the training data. It might produce inaccurate or unexpected outputs for such cases.
+    Morphosyntactically Complex Cases: Items requiring rare or non-typical patterns of forming personatives can pose challenges for the model. The accuracy of the generated output may decrease in such scenarios.
+    Feminine Nouns: The model has shown to sometimes underperform when dealing with feminine nouns, potentially due to biases or patterns in the training data. Further investigation and fine-tuning are needed to address this limitation.
+    Single Sentence Input: The model is optimized for single-sentence inputs and might not produce the desired results for single-word items, longer texts or paragraphs. It might fail to maintain context, coherence and terminological consistency across multiple sentences. Its performance for single-word items has not been tested.
+    Domain Specificity: The model is trained on a specific dataset of single sentences with job titles and without them. It may not generalize well to other domains or contexts. It might need further fine-tuning to adapt to different types of text or specific vocabulary.
 
+    More information regarding issues, errors and limitations, see our [readme](https://github.com/ArielUW/IMLLA-FinalProject/blob/main/README.md).
 ## Training and evaluation data
 
-The model's performance has been evaluated in terms of neutralization attempt precision and recall, as well as Levenshtein distance from gold-standard items. More information on the evaluation outcomes can be found in [our readme](https://github.com/ArielUW/IMLLA-FinalProject/blob/main/README.md).
+This model was evaluated using several metrics to assess its performance:
+
+    BLEU (Bilingual Evaluation Understudy): BLEU is a widely used metric for evaluating machine translation quality. It measures the overlap between the generated text and the reference text in terms of n-grams. A higher BLEU score indicates better translation quality. The model achieved a BLEU score of 93.9441 on the evaluation set, indicating high accuracy in generating gender-neutral terms.
+    Attempted Noun Neutralisation Precision: This metric measures the proportion of correctly attempted neutralizations (i.e., items that required neutralization, not necessarily correctly formed neutral items) out of all attempted neutralizations. The model achieved a precision of 1, indicating that all attempted neutralizations were performed on items that required it.
+    Attempted Noun Neutralisation Recall: This metric measures the proportion of nouns that had a neutralization attempt present in the generated text out of all nouns that should have been neutralized. The model achieved a recall of 0.892, suggesting that it successfully recognized items requiring neutralization in the majority of cases.
+    Normalized Levenshtein's Distance: This metric calculates the edit distance between the generated text and the reference text, normalized by the length of the reference text. It provides a measure of similarity between the two texts. The model achieved a Levenshtein's distance of 0.0395 for sentences requiring neutralization and 0.0001 for the items that should not have been changed at all, indicating a high degree of similarity between the generated text and the reference text.
+
+More information on the evaluation outcomes can be found in [our readme](https://github.com/ArielUW/IMLLA-FinalProject/blob/main/README.md).
 
 ## Training procedure