MathBite/self_corrective_llama_3.1_8B_untrained

This is a version of meta-llama/Llama-3.1-8B-Instruct modified with a custom architecture to support self-correction via hallucination detection.

This model, an instance of SelfCorrectiveLlama, includes a hallucination detection head that modifies the logits of special tokens to aid in content generation and revision.

Special Tokens

The tokenizer has been expanded to include the following special tokens: <DEL_W>, <DEL_S>, <DEL_A>.

How to Use

Because this model uses a custom architecture, you must use trust_remote_code=True when loading it. The required modeling.py file is included in this repository.

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "MathBite/self_corrective_llama_3.1_8B_untrained"
tokenizer = AutoTokenizer.from_pretrained(model_name)

# Important: You must trust the remote code
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    trust_remote_code=True
)

# You can now use the model for generation
# For example, to get hallucination probabilities:
# (sequences, p_halls) = model.generate(..., output_p_hall=True)

Model Details

This model was programmatically converted and uploaded using a deployment script. The custom class SelfCorrectiveLlama can be found in the modeling.py file.

The code in modeling.py is licensed under the Apache 2.0 License. The model weights are subject to the original license of the base model.