ANK-fv1

This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model has been fine-tuned for mathematical reasoning tasks.

Model Details

Model Description

• Developed by: eastbrick
• Shared by : eastbrick
• Model type: Causal Language Model (Merged)
• Language(s) (NLP): English
• License: Apache 2.0
• Finetuned from model: microsoft/Phi-4

Uses

Direct Use

This model can be used for mathematical reasoning and problem-solving tasks. It specializes in step-by-step mathematical problem solving and educational assistance.

Downstream Use [optional]

Can be used for various mathematical and reasoning applications including:

• Mathematical problem solving
• Step-by-step reasoning
• Educational math assistance
• Complex mathematical calculations

Bias, Risks, and Limitations

This model inherits the biases and limitations of the base Phi-4 model. Users should be aware of:

• Potential generation of biased or harmful content
• Hallucination in factual information
• Performance may vary on different types of prompts
• Mathematical reasoning accuracy may vary with problem complexity

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model.

• Use appropriate safety filtering for production applications
• Validate mathematical solutions generated by the model
• Consider the model's training data and potential biases
• Test thoroughly on your specific use case

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "eastbrick/ank-fv1",
    torch_dtype=torch.float16,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("eastbrick/ank-fv1")

# Prepare input
prompt = "What is the integral of x^3 + 2x^2 - 5x + 3?"#example question
inputs = tokenizer(prompt, return_tensors="pt")

# Generate
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=256,#can be kept anything based on the input
        do_sample=True,
        pad_token_id=tokenizer.eos_token_id
    )

# Decode response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Training Details

Training Data

This model was fine-tuned on a filtered subset of the DeepMath-103k dataset, containing approximately 52,000 harder mathematical problems selected for increased difficulty and complexity.

Training Procedure

Preprocessing [optional]

The data was filtered based on difficulty level of the questions in order to take only quality questions which can be challenging to solve specifically for phi-4.

Training Hyperparameters

• Training regime: LoRA (Low-Rank Adaptation) fine-tuning
• Base model: Phi-4 14B
• Training time: ~12 hours
• Final training loss: ~0.40
• Framework: Transformers + PEFT
• Training method: LoRA fine-tuning with subsequent merge

Speeds, Sizes, Times [optional]

• Training time: 12 hours
• Hardware: NVIDIA RTX 4090
• Model size: ~14B parameters
• Training dataset size: 52,000 mathematical problems

Evaluation

• 35% improvement in accuracy on top of base model in the AIME-2025 questions
• Better step by step solutions.

Technical Specifications [optional]

Model Architecture and Objective

This is a fully merged model with the following specifications:

• Model size: ~14B parameters
• Specialization: Mathematical reasoning and problem-solving
• Base architecture: Phi-4 14B
• Context length: 16384 tokens

Compute Infrastructure

Hardware

• Training: NVIDIA RTX 4090
• Power consumption: ~450W average during training
• Total energy: ~5.4 kWh

Software

• Framework: PyTorch + Transformers + PEFT
• Training: LoRA fine-tuning framework

Citation [optional]

BibTeX:

@misc{ank-v1-2025,
  title={ANK-v1: Fine-tuned Phi-4 14B with LoRA for Mathematical Reasoning},
  author={eastbrick},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/eastbrick/ank-fv1}}
}

APA:

eastbrick. (2025). ANK-fv1: Fine-tuned Phi-4 14B with LoRA for Mathematical Reasoning. Hugging Face. https://huggingface.co/eastbrick/ank-v1

Model Card Authors [optional]

eastbrick

---

This model card was generated to provide transparency about the model's capabilities, limitations, and intended use cases.