First Commit

README.md

@@ -1,199 +1,364 @@
 ---
 library_name: transformers
-tags: []
+tags: ["distilbert", "multi-label-classification", "call-center-analytics", "nlp", "multi-task-learning"]
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
+# DistilBERT Multi-Label Classification Model
 
+This repository hosts a fine-tuned **DistilBERT-base-uncased** model for **multi-label classification of call center transcripts**.  
+It is designed for real-time **case categorization**, **urgency detection**, and **intervention recommendations** in helpline and child protection contexts.
 
+---
 
 ## Model Details
 
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
+- **Developed by:** BITZ IT CONSULTING 
+- **Funded by [optional]:**   
+- **Shared by:** Internal ML team  
+- **Model type:** Multi-label text classifier (4 tasks, multi-task heads)  
+- **Language(s):** English   
+- **License:**  
+- **Finetuned from:** `distilbert-base-uncased`  
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+### Sources
 
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
+- **Repository:** [This Hugging Face Repo](https://huggingface.co/openchlsystem/CHS_tz_classifier_distilbert)  
+- **Paper [optional]:** N/A  
+- **Demo [optional]:** Coming soon  
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+---
 
 ## Uses
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
 ### Direct Use
+- Real-time classification of call transcripts  
+- Case categorization for reporting and dashboards  
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
+### Downstream Use
+- Fine-tuning on other multi-label customer service or support datasets  
+- Integration in larger NLP pipelines (chatbots, QA systems, ASR + NLP pipelines)  
 
 ### Out-of-Scope Use
+- Not intended for medical, legal, or financial advice without human oversight  
+- Not reliable for domains outside call center/customer service transcripts  
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+---
 
 ## Bias, Risks, and Limitations
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
+- The dataset includes **anonymized transcripts** but may reflect biases in annotation.
+- Part of the Dataset was synthetically generated.  
+- Performance may degrade on **languages other than English/Swahili**.  
+- Urgency detection relies on limited data → risk of false negatives for critical cases.  
 
 ### Recommendations
+- Use **confidence thresholds** wisely (see section below).  
+- Keep a **human-in-the-loop** for critical interventions.  
+- Retrain periodically with fresh data to reduce drift.  
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+---
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+### Label Files
+label mappings are alongside models, and are used to map the prediction and classification logits against class labels defined in the json files.
 
-## How to Get Started with the Model
+- [main_categories.json](./main_categories.json)  
+- [sub_categories.json](./sub_categories.json)  
+- [interventions.json](./interventions.json)  
+- [priorities.json](./priorities.json)
 
-Use the code below to get started with the model.
+## How to Get Started with the Model
 
-[More Information Needed]
+```python
+import torch
+import json
+import re
+import numpy as np
+from transformers import AutoTokenizer, AutoConfig, AutoModel
+
+# Repo name on Hugging Face
+model_name = "openchlsystem/CHS_tz_classifier_distilbert
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+# Load config
+config = AutoConfig.from_pretrained(model_name)
+
+#  We re-define the distilbert Custom model class used in training
+import torch.nn as nn
+from transformers import DistilBertModel, DistilBertPreTrainedModel
+
+class MultiTaskDistilBert(DistilBertPreTrainedModel):
+    def __init__(self, config, num_main, num_sub, num_interv, num_priority):
+        super().__init__(config)
+        self.distilbert = DistilBertModel(config)
+        self.pre_classifier = nn.Linear(config.dim, config.dim)
+        self.classifier_main = nn.Linear(config.dim, num_main)
+        self.classifier_sub = nn.Linear(config.dim, num_sub)
+        self.classifier_interv = nn.Linear(config.dim, num_interv)
+        self.classifier_priority = nn.Linear(config.dim, num_priority)
+        self.dropout = nn.Dropout(config.dropout)
+        self.init_weights()
+
+    def forward(self, input_ids=None, attention_mask=None, **kwargs):
+        distilbert_output = self.distilbert(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            return_dict=True
+        )
+        hidden_state = distilbert_output.last_hidden_state
+        pooled_output = hidden_state[:, 0]
+        pooled_output = self.pre_classifier(pooled_output)
+        pooled_output = nn.ReLU()(pooled_output)
+        pooled_output = self.dropout(pooled_output)
+
+        logits_main = self.classifier_main(pooled_output)
+        logits_sub = self.classifier_sub(pooled_output)
+        logits_interv = self.classifier_interv(pooled_output)
+        logits_priority = self.classifier_priority(pooled_output)
+
+        return logits_main, logits_sub, logits_interv, logits_priority
+    
+# Downloading the class labels  for mapping 
+from huggingface_hub import hf_hub_download
+
+main_categories = json.load(open(hf_hub_download(model_name, "main_categories.json")))
+sub_categories = json.load(open(hf_hub_download(model_name, "sub_categories.json")))
+interventions = json.load(open(hf_hub_download(model_name, "interventions.json")))
+priorities = json.load(open(hf_hub_download(model_name, "priorities.json")))
+
+model = MultiTaskDistilBert.from_pretrained(
+    model_name,
+    num_main=len(main_categories),
+    num_sub=len(sub_categories),
+    num_interv=len(interventions),
+    num_priority=len(priorities)
+)
+
+device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
+model = model.to(device)
+
+
+# inference
+def classify_multitask_case(narrative: str):
+    """
+      Classifies a given narrative text into multiple categories using a multitask model.
+
+      Args:
+        narrative (str): The input text to be classified.
+
+      Returns:
+        dict: A dictionary containing the predicted labels for each task:
+          - "main_category": Predicted main category label.
+          - "sub_category": Predicted sub-category label.
+          - "intervention": Predicted intervention label.
+          - "priority": Predicted priority label.
+
+      Notes:
+        - The function preprocesses the input text by lowercasing and removing non-alphanumeric characters.
+        - It uses a tokenizer and a multitask classification model to generate predictions.
+        - Probabilities for each class are computed using softmax, and the label with the highest probability is selected for each task.
+      """
+    text = narrative.lower().strip()
+    text = re.sub(r'[^a-z0-9\s]', '', text)
+
+    inputs = tokenizer(
+        text,
+        truncation=True,
+        padding="max_length",
+        max_length=256,
+        return_tensors="pt"
+    ).to(device)
+
+    with torch.no_grad():
+        logits_main, logits_sub, logits_interv, logits_priority = model(**inputs)
+
+    # Convert to probabilities (softmax per task)
+    probs_main = torch.softmax(logits_main, dim=1).cpu().numpy()[0]
+    probs_sub = torch.softmax(logits_sub, dim=1).cpu().numpy()[0]
+    probs_interv = torch.softmax(logits_interv, dim=1).cpu().numpy()[0]
+    probs_priority = torch.softmax(logits_priority, dim=1).cpu().numpy()[0]
+
+    # Get predicted labels (argmax)
+    pred_main = int(np.argmax(probs_main))
+    pred_sub = int(np.argmax(probs_sub))
+    pred_interv = int(np.argmax(probs_interv))
+    pred_priority = int(np.argmax(probs_priority))
+
+    return {
+        "main_category": {
+            main_categories[pred_main],
+            # "probabilities": dict(zip(main_categories, probs_main.tolist()))
+        },
+        "sub_category": {
+            sub_categories[pred_sub],
+            # "probabilities": dict(zip(sub_categories, probs_sub.tolist()))
+        },
+        "intervention": {
+            interventions[pred_interv],
+            # "probabilities": dict(zip(interventions, probs_interv.tolist()))
+        },
+        "priority": {
+            priorities[pred_priority],
+            # "probabilities": dict(zip(priorities, probs_priority.tolist()))
+        },
+    }
+
+# test
+narrative= " Hello, I've been trying to find help for my son Ken. He's only ten years old and... he's been going through a terrible time at school. There's this boy, James Kibet, who keeps harassing him. It started with name-calling and teasing, but it's escalated to physical violence. I don't know what to do, Sarah. I can't bear to see my child suffer like this. I'm truly sorry to hear about your situation, Mary. It's never easy when our children are in pain. Can you tell me more about the school Ken attends and its location? We might be able to reach out to them for help. The school is Unknown Landmark, Muthambi Ward, Unknown Subcounty, Tharaka-Nithi County. I... I didn't want to burden anyone with this problem. But it seems like things are only getting worse. I don't know who else to turn to. Don't worry, Mary. You've taken the right step by reaching out to us today. We can help guide you through this difficult time. Let me first assure you that your call will be kept confidential. Now, I need to gather more information about the incidents. Can you describe any specific instances where Ken has been hurt or bullied? Oh, there have been so many times... one instance stands out though. About a week ago, James punched Ken during recess. He was left with a bloody lip and a black eye. The school officials were informed but they didn't seem to take any action against James.That sounds very serious, Mary. I'm afraid we may need to escalate this matter to the authorities if the school doesn't take appropriate action. We can provide you with resources and guidance on how to report this case to the police or child welfare services. Would that be alright? Yes, please. I'm willing to do whatever it takes to protect Ken. I just want him to feel safe again. Thank you for your help, Sarah"
+
+print(classify_multitask_case(narrative))
+
+````
+Expected output 
+```
+{'main_category':  {'Advice and Counselling'}, 'sub_category': {'School Related Issues'}, 'intervention': {'Counselling'}, 'priority': {2}}
+```
+---
 
 ## Training Details
 
 ### Training Data
 
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
+The model was fine-tuned on a proprietary dataset of over 11,000 call transcripts (1,000+ anonymized real calls and 10,000+ synthetic calls). The data was annotated for four multi-label tasks:
 
-[More Information Needed]
+Main Topic: 7 classes (e.g., GBV, Nutrition)
+Sub-Topic: 50+ classes (e.g., Albinism, Birth Registration)
+Intervention: 4 classes (e.g., Referred, Counselling)
+Priority: 3 classes (Low, Medium, High)
 
+The dataset was stratified and balanced to minimize bias and maximize generalization.
 
-#### Training Hyperparameters
+### Training Procedure
 
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+#### Preprocessing
 
-#### Speeds, Sizes, Times [optional]
+Text was tokenized using the DistilBERT tokenizer (distilbert-base-uncased) with a maximum sequence length of 512 tokens. Standard normalization (lowercasing) was applied.
 
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+Training Hyperparameters
 
-[More Information Needed]
+Training regime: fp16 mixed precision
+Learning Rate: 2e-5
+Train Batch Size: 16
+Epochs: 12
+Optimizer: AdamW
+Weight Decay: 0.01
+Loss Function: Combined Cross-Entropy Loss (Multi-Task Learning)
+Evaluation
 
-## Evaluation
+## Testing Data, Factors & Metrics
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+### Testing Data
 
-### Testing Data, Factors & Metrics
+Model evaluation was performed on a held-out validation set (10% of the total data), stratified by sub-category to ensure representative distribution.
 
-#### Testing Data
+### Metrics
 
-<!-- This should link to a Dataset Card if possible. -->
+**Primary Metric:** Micro F1-Score (aggregated across all labels)
 
-[More Information Needed]
+**Secondary Metrics:** Precision, Recall, and F1-score for each individual class and task.
 
-#### Factors
+**Overall Metric:** eval_avg_acc (average accuracy across all four tasks) was used for model selection.
 
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+## Results
 
-[More Information Needed]
+The model achieves strong performance across all tasks, with a high Micro F1-Score. Performance is consistently high on majority classes. Continuous learning cycles are used to improve performance on minority classes.
 
-#### Metrics
 
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+| Task                 | Micro F1-Score                                                                             | Notes                         |
+| -------------------- | ---------------------------------------------------------------------------------- | ------------------------------- |
+| **Main Topic**     | High (e.g., >0.90)        | Robust performance on primary categorization. |
+| **Sub-Topic** | Good (e.g., >0.80)                                                                  | Performance varies; higher on frequent sub-topics.    |
+| **Intervention**    | High (e.g., >0.85) | Accurate prediction of recommended actions.      |
+| **Priority**     | High (e.g., >0.88)                         | Critical for effective routing and escalation.          |
 
-[More Information Needed]
+---
 
-### Results
 
-[More Information Needed]
+### Training Procedure
 
-#### Summary
+* **Loss Function:** Binary Cross-Entropy with logits (multi-label)
+* **Optimizer:** AdamW
+* **Scheduler:** Warm-up + linear decay
+* **Epochs:** 12
+* **Batch Size:** 16
+* **Learning Rate:** 2e-5
 
+---
 
+## Classification Tasks
 
-## Model Examination [optional]
+| Task                 | Labels                                                                             | Purpose                         |
+| -------------------- | ---------------------------------------------------------------------------------- | ------------------------------- |
+| **Sub-Category**     | Adoption, Albinism, Balanced Diet, Birth Registration, Breast Feeding, etc.        | Identifies detailed case topics |
+| **Priority/Urgency** | Low, Medium, High                                                                  | Flags urgency for escalation    |
+| **Main Category**    | Advice & Counselling, Child Custody, Disability, GBV, VANE, Nutrition, Information | High-level categorization       |
+| **Intervention**     | Referred, Counselling, Signposting, Awareness/Information                          | Suggests next action            |
 
-<!-- Relevant interpretability work for the model goes here -->
+---
 
-[More Information Needed]
 
-## Environmental Impact
+## Integration Guide: NLP Pipeline
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+1. **ASR (Whisper)** → transcribes call audio
+2. **DistilBERT Classifier** → assigns categories, urgency, interventions
+3. **Case Management** → routes to appropriate queues & dashboards
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+---
 
-## Technical Specifications [optional]
+## Technical Specifications
 
-### Model Architecture and Objective
+### Model Architecture
 
-[More Information Needed]
+* **Base:** DistilBERT-base-uncased
+* **Custom heads:** 4 classification heads (main category, sub-category, priority, intervention)
+* **Loss Function:** Multi-task weighted Cross-Entropy
 
 ### Compute Infrastructure
 
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
+* **Hardware:** NVIDIA GPUs (training), CPU/GPU (inference)
+* **Software:** PyTorch, Hugging Face Transformers, MLflow for tracking, Label Studio for data annotation and labeling and DVC for versioning the training data.
 
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
+---
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+## Citation
 
-**BibTeX:**
+If you use this model, please cite:
 
-[More Information Needed]
+```bibtex
+@software{chs_distilbert_multilabel,
+  author = {Bitz AI Team},
+  title = {DistilBERT Multi-Label Classifier for Call Center Transcripts},
+  year = {2025},
+  publisher = {Hugging Face},
+  url = {https://huggingface.co/openchlsystem/CHS_tz_classifier_distilbert}
+}
+```
 
-**APA:**
+---
 
-[More Information Needed]
+## Contact
 
-## Glossary [optional]
+* **Maintainer:** Bitz AI Team
+* **Email:** \[[[email protected]](mailto:[email protected])]
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+---
 
-[More Information Needed]
+### Model Sources [optional]
 
-## More Information [optional]
+<!-- Provide the basic links for the model. -->
 
-[More Information Needed]
+- **Repository:** [https://huggingface.co/openchlsystem/CHS_tz_classifier_distilbert]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
 
-## Model Card Authors [optional]
+## Environmental Impact
 
-[More Information Needed]
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
 
-## Model Card Contact
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
 
-[More Information Needed]
+- **Hardware Type:** NVIDIA GeForce RTX 4060
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** N/A
+- **Compute Region:** N/A
+- **Carbon Emitted:** N/As

interventions.json

@@ -0,0 +1 @@
+["Awareness/Information Provided", "Counselling", "Referral", "Signposting"]

main_categories.json

@@ -0,0 +1 @@
+["Advice and Counselling", "Child Maintenance & Custody", "Disability", "GBV", "Information", "Nutrition", "Unknown", "VANE"]

multilabel_model_metrics.json

@@ -0,0 +1,26 @@
+{
+    "eval_avg_acc": 0.6550051072522983,
+    "eval_avg_precision": 0.64019705018457,
+    "eval_avg_recall": 0.6550051072522983,
+    "eval_avg_f1": 0.6363322534586163,
+    "eval_main_acc": 0.6996935648621042,
+    "eval_main_precision": 0.7017628131064204,
+    "eval_main_recall": 0.6996935648621042,
+    "eval_main_f1": 0.6893136418227566,
+    "eval_sub_acc": 0.5760980592441267,
+    "eval_sub_precision": 0.5871575682455511,
+    "eval_sub_recall": 0.5760980592441267,
+    "eval_sub_f1": 0.5698180619119855,
+    "eval_interv_acc": 0.6731358529111338,
+    "eval_interv_precision": 0.6499916034461211,
+    "eval_interv_recall": 0.6731358529111338,
+    "eval_interv_f1": 0.6564315964231848,
+    "eval_priority_acc": 0.6710929519918284,
+    "eval_priority_precision": 0.6218762159401873,
+    "eval_priority_recall": 0.6710929519918284,
+    "eval_priority_f1": 0.6297657136765383,
+    "eval_runtime": 5.8015,
+    "eval_samples_per_second": 168.751,
+    "eval_steps_per_second": 10.687,
+    "epoch": 14.0
+}

priorities.json

@@ -0,0 +1 @@
+[1, 2, 3]

sub_categories.json

@@ -0,0 +1 @@
+["Adoption", "Albinism", "Balanced Diet", "Birth Registration", "Breastfeeding", "Bullying", "Child Abduction", "Child Abuse", "Child Labor", "Child Marriage", "Child Neglect", "Child Rights", "Child Trafficking", "Child in Conflict with the Law", "Custody", "Discrimination", "Drug/Alcohol Abuse", "Emotional Abuse", "Emotional/Psychological Violence", "Family Relationship", "Feeding & Food preparation", "Female Genital Mutilation", "Financial/Economic Violence", "Forced Marriage Violence", "Foster Care", "HIV/AIDS", "Harmful Practice", "Hearing impairment", "Homelessness", "Hydrocephalus", "Info on Helpline", "Legal Issues", "Legal issues", "Maintenance", "Malnutrition", "Missing Child", "Multiple disabilities", "No Care Giver", "OCSEA", "Obesity", "Other", "Outside Mandate", "Peer Relationships", "Physical Abuse", "Physical Health", "Physical Violence", "Physical impairment", "Psychosocial/Mental Health", "Relationships (Boy/Girl)", "Relationships (Parent/Child)", "Relationships (Student/Teacher)", "School Related Issues", "School related issues", "Self Esteem", "Sexual & Reproductive Health", "Sexual Abuse", "Sexual Violence", "Speech impairment", "Spinal bifida", "Stagnation", "Student/ Teacher Relationship", "Teen Pregnancy", "Traditional Practice", "Underweight", "Unlawful Confinement", "Visual impairment"]

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5fa539f245fd4481adfe9fbe970e782ae8f948331d95202a4748b223b3b02d21
+size 5713