aryan27/geometry-model-hf

MiniLLM Geometry Engine

MiniLLM Geometry Engine is a fine-tuned version of the TinyLlama 1.1B language model, optimized for generating geometry-related functions for a geometry engine. Fine-tuned on a custom Geometry Chain-of-Thought (CoT) dataset, this model excels at producing accurate and efficient mathematical functions for geometric computations, stored in the Hugging Face .safetensors format for secure and efficient model loading.

Model Overview

• Base Model: TinyLlama 1.1B

• Fine-Tuning Dataset: Custom Geometry CoT dataset consisting of coordinate geometry problems and geometrical construction instruction

• Model Format: .safetensors for secure and efficient weight storage

• Intended Use: Generating geometry functions for educational tools.

• License: MIT

• Contact: Contact via Hugging Face profile

Important Note:

• Use the system_prompt.txt for the System Prompt, this provides the accurate results.

• You can extract and edit the system prompt. I hope to add new functions later.

Capabilities

MiniLLM Geometry Engine interprets geometry-related queries and outputs functional Python code or mathematical expressions. For example, given a prompt like "Generate a function to calculate the area of a triangle," it produces executable code or formulas with clear reasoning, leveraging its CoT fine-tuning for logical accuracy. Supported tasks include area, volume, distance, and intersection calculations.

Technical Details

• Architecture: Transformer-based, inherited from TinyLlama 1.1B

• Fine-Tuning Details: Trained on a dataset of geometry problems with step-by-step solutions

• Output Format: Python code or pseudocode compatible with geometry engine APIs

• Performance: Improved accuracy on geometry tasks compared to the base TinyLlama model, with low latency

• Model Storage: Uses .safetensors for secure and efficient loading with Hugging Face's safetensors library

Usage

To load and use the model with Hugging Face's Transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer

from safetensors.torch import load_file

# Load tokenizer and model

tokenizer = AutoTokenizer.from_pretrained("aryan27/geometry-model-hf")

model = AutoModelForCausalLM.from_pretrained("aryan27/geometry-model-hf")

# Example prompt

prompt = "Draw a point A at 3,4"

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(*inputs, max_length=200)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Install required libraries:

pip install safetensors transformers

Training Details

• Dataset: Custom Geometry CoT dataset with 10,000 geometry problems and solutions

• Training Procedure: Fine-tuned for 3 epochs on a single NVIDIA A100 GPU

• Hyperparameters: Learning rate: 2e-5, batch size: 16

• Hardware: NVIDIA A100 GPU

Limitations

• Supports very few functions since this was an experimental model, maybe I will add other functions later.

• Limited to geometry-related tasks and may not generalize to other mathematical domains.

• Extensive use of strict System Prompting. I aim to eliminate that also.

How to Contribute

Submit issues or pull requests via the Hugging Face repository. Contributions to the dataset or model improvements are welcome.

Acknowledgments

Thanks to the TinyLlama team for the base model and Hugging Face for the safetensors and transformers libraries.

MiniLLM Geometry Engine is a lightweight, efficient solution for developers and researchers needing reliable geometry function generation, with the security and performance benefits of the .safetensors format.