Model Card for Model ID

This model is a fine-tuned RoBERTa-based classifier designed to predict the presence of ten moral foundations (five virtues and five vices) within text. It can be used for various text analysis tasks, such as content analysis, opinion mining.

Model Details

Model Description

Developed by: M. Murat Ardag
Funded by [optional]: not relevant
Shared by [optional]: not relevant
Model type: Multi-label Classification
License: GPL-3.0
Finetuned from model [optional]: roberta-base

Model Sources [optional]

Repository: https://huggingface.co/MMADS/MoralFoundationsClassifier
See the following sources & papers for alternatives:
- https://moralfoundations.org/other-materials/
- Frimer, J. A., Boghrati, R., Haidt, J., Graham, J., & Dehgani, M. (2019). Moral Foundations Dictionary for Linguistic Analyses 2.0. Unpublished manuscript.
- Hopp FR, Fisher JT, Cornell D, Huskey R, Weber R. The extended Moral Foundations Dictionary (eMFD): Development and applications of a crowd-sourced approach to extracting moral intuitions from text. Behav Res Methods. 2021 Feb;53(1):232-246. doi: 10.3758/s13428-020-01433-0. PMID: 32666393.
- Kennedy B, Atari M, Mostafazadeh Davani A, Hoover J, Omrani A, Graham J, Dehghani M. Moral concerns are differentially observable in language. Cognition. 2021 Jul;212:104696. doi: 10.1016/j.cognition.2021.104696. Epub 2021 Mar 31. PMID: 33812153.

Uses

Direct Use

The model can be directly used for classifying text into the following moral foundations:

Care: Care/harm for others, protecting them from harm.

Fairness: Justice, treating others equally.

Loyalty: Group loyalty, patriotism, self-sacrifice for the group.

Authority: Respect for tradition and legitimate authority.

Sanctity: Disgust, avoiding dangerous diseases and contaminants.

Each foundation is represented as a virtue (positive expression) and a vice (negative expression).

It's particularly useful for researchers, policymakers, and analysts interested in understanding moral reasoning and rhetoric in different contexts.

Downstream Use

Potential downstream uses include:

Content analysis: Analyzing the moral framing of news articles, social media posts, or other types of text.

Opinion mining: Understanding the moral values underlying people's opinions and arguments.

Ethical assessment: Evaluating the ethical implications of decisions, policies, or products.

Out-of-Scope Use

This model is not designed for predicting specific actions or behaviors based on moral foundations.
It may not accurately generalize to texts that are significantly different in style or domain from its training data (NOT RECOMMENDED FOR SOCIAL MEDIA DATA)
The model should not be used to make definitive judgments about the morality of individuals or groups.

Bias, Risks, and Limitations

The model is trained on a diverse corpus but may still reflect biases present in the data.
It may be sensitive to specific wording or phrasing, leading to inconsistent predictions.
The interpretation of moral foundations can be subjective, and the model's predictions may not always align with human judgment.

Recommendations

Be aware of the potential biases and limitations of the model.
Carefully consider the context and purpose of the text being analyzed.
Use the model's predictions as a starting point for further analysis and discussion.

How to Get Started with the Model

Use the code below to get started with the model.

# How to Get Started with the Model

import torch
from transformers import RobertaTokenizer, RobertaForSequenceClassification

# Load the model and tokenizer
model_path = "MMADS/MoralFoundationsClassifier"
model = RobertaForSequenceClassification.from_pretrained(model_path)
tokenizer = RobertaTokenizer.from_pretrained(model_path)

# Define label names based on Moral Foundations Theory
# Each foundation has a virtue (positive) and vice (negative) dimension
label_names = [
    "care_virtue",      # Compassion, kindness, nurturing
    "care_vice",        # Harm, cruelty, suffering
    "fairness_virtue",  # Justice, equality, reciprocity
    "fairness_vice",    # Cheating, inequality, injustice
    "loyalty_virtue",   # Loyalty, patriotism, self-sacrifice
    "loyalty_vice",     # Betrayal, treason, disloyalty
    "authority_virtue", # Respect, tradition, order
    "authority_vice",   # Subversion, disobedience, chaos
    "sanctity_virtue",  # Purity, sanctity, nobility
    "sanctity_vice"     # Degradation, contamination, impurity
]

# Function to make predictions
def predict_moral_foundations(texts, threshold=0.65):
    """
    Predict moral foundations present in a batch of texts.
    
    Args:
        texts (list of str): A list of input texts to analyze.
        threshold (float): Probability threshold for positive prediction (default: 0.65).
    
    Returns:
        list of dict: A list of dictionaries, one for each input text.
    """
    # Tokenize and prepare input
    # The tokenizer handles a list of strings automatically, creating a batch.
    inputs = tokenizer(texts, return_tensors="pt", truncation=True, 
                       padding=True, max_length=512)
    
    # Move to GPU if available
    device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
    model.to(device)
    inputs = {k: v.to(device) for k, v in inputs.items()}
    
    # Get predictions
    with torch.no_grad():
        outputs = model(**inputs)
        logits = outputs.logits
        probabilities = torch.sigmoid(logits)
    
    # Format results for the entire batch
    all_results = []
    batch_probs = probabilities.cpu().numpy()
    
    for single_text_probs in batch_probs:
        results = {}
        for i, label in enumerate(label_names):
            results[label] = {
                "probability": float(single_text_probs[i]),
                "predicted": bool(single_text_probs[i] > threshold)
            }
        all_results.append(results)
    
    return all_results

# Example usage with a list of texts
texts = [
    "You don't actually believe what you're saying.",
    "People are calling you stupid but you're just good old fashioned lying.",
    "Even if you've never held employment in your life there is no way you think employers just hand out sick days whenever their employees feel like it.",
    "Troll on."
]
all_predictions = predict_moral_foundations(texts)

# Display detected foundations for each text
for i, text in enumerate(texts):
    print(f"Analyzing: '{text}'")
    print("Detected moral foundations:")
    
    predictions = all_predictions[i]
    detected_foundations = False
    for foundation, data in predictions.items():
        if data['predicted']:
            print(f"  - {foundation}: {data['probability']:.3f}")
            detected_foundations = True
    
    if not detected_foundations:
        print("  - None")
        
    print("-" * 30) # Separator for clarity

Training Details

Training Data

The model was trained on a diverse corpus of text, including:

Personal blogs
Political blogs
News media articles
Essays
Short stories
Parliamentary debates from US, UK, CA, NZ
Speeches at the UN
Speeches by central bank presidents

The trainng data is a subset of >60M sentences.

Training Procedure

The model was fine-tuned using the HuggingFace Transformers library with the following hyperparameters:

Num_train_epochs: 10
Per_device_train_batch_size: 8
Per_device_eval_batch_size: 8
Learning rate: 3e-5
Optimizer: AdamW
Loss function: Binary Cross Entropy with Logits Loss

Evaluation

Testing Data, Factors & Metrics

Testing Data

The model was evaluated on a held-out portion of the training data.

Metrics

Overall Accuracy: 0.9957
Overall Precision: 0.9957
Overall Recall: 0.9957
Overall F1-score: 0.9957

See the section below for detailed per-class metrics.

Results

The model achieves high overall performance, with variations across different moral foundations. The "loyalty_vice" category has a notably lower F1-score due to low recall, indicating difficulty in identifying this specific vice.

Per-class metrics:

care_virtue: accuracy: 0.9954 precision: 0.9779 recall: 0.9758 f1: 0.9769
care_vice: accuracy: 0.9960 precision: 0.9734 recall: 0.9506 f1: 0.9619
fairness_virtue: accuracy: 0.9974 precision: 0.9786 recall: 0.9645 f1: 0.9715
fairness_vice: accuracy: 0.9970 precision: 0.9319 recall: 0.8574 f1: 0.8931
loyalty_virtue: accuracy: 0.9945 precision: 0.9811 recall: 0.9780 f1: 0.9795
loyalty_vice: accuracy: 0.9972 precision: 1.0000 recall: 0.0531 f1: 0.1008
authority_virtue: accuracy: 0.9914 precision: 0.9621 recall: 0.9683 f1: 0.9652
authority_vice: accuracy: 0.9963 precision: 0.9848 recall: 0.5838 f1: 0.7331
sanctity_virtue: accuracy: 0.9963 precision: 0.9640 recall: 0.9458 f1: 0.9548
sanctity_vice: accuracy: 0.9958 precision: 0.9538 recall: 0.8530 f1: 0.9006

Model Examination

Environmental Impact

Minimal Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

Hardware Type: Google Colab GPU
Hours used: 8
Cloud Provider: Google
Compute Region: South Carolina

Technical Specifications

Model Architecture and Objective

The model is based on the RoBERTa architecture, fine-tuned for multi-label classification. It uses a custom loss function (BCEWithLogitsLoss) to handle the multi-label nature of the task.

More Information

This model is based on Moral Foundations Theory, a social psychological theory that explains the origins and variation in human moral reasoning. The theory proposes six moral foundations: Care/Harm, Fairness/Cheating, Loyalty/Betrayal, Authority/Subversion, Sanctity/Degradation, and Liberty/Oppression.

Though, the current model focuses on the first five.

Model Card Authors

M. Murat Ardag

Model Card Contact

via my personal website. thx

Citation

If you use this model in your research or applications, please cite it as follows:

Ardag, M.M. (2024) Moral Foundations Classifier. HuggingFace. https://doi.org/10.57967/hf/2774

Glossary

Moral Foundations Theory identifies five core foundations guiding human morality across cultures:

Care/Harm: Rooted in empathy and attachment, it values kindness and compassion.
Fairness/Cheating: Based on reciprocal altruism, it emphasizes justice and fairness.
Loyalty/Betrayal: Stemming from humans' tribal past, it values group loyalty and condemns betrayal.
Authority/Subversion: Shaped by hierarchical social structures, it values respect for authority and tradition.
Sanctity/Degradation: Linked to disgust and purity, it values living in a noble way and avoiding degradation.

MMADS
/

MoralFoundationsClassifier