Triangle104/Athena-3-7B-Q5_K_M-GGUF

This model was converted to GGUF format from Spestly/Athena-3-7B using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model.

---

Athena-3-7B is a 7.68-billion-parameter causal language model fine-tuned from Qwen2.5-Math-7B. This model is designed to excel in STEM reasoning, mathematics, and natural language processing tasks, offering advanced instruction-following and problem-solving capabilities.

Training Details

Athena-3-7B was fine-tuned using the Unsloth framework on a single NVIDIA A100 GPU. The fine-tuning process spanned approximately 90 minutes over 60 epochs, utilizing a curated dataset focused on instruction-following, problem-solving, and advanced mathematics. This approach enhances the model's capabilities in academic and analytical tasks.

Intended Use

Athena-3-7B is designed for a range of applications, including but not limited to:

-STEM Reasoning: Assisting with complex problem-solving and theoretical explanations.

-Academic Assistance: Supporting tutoring, step-by-step math solutions, and scientific writing.

-General NLP Tasks: Text generation, summarization, and question answering.

-Data Analysis: Interpreting and explaining mathematical and statistical data.

While Athena-3-7B is a powerful tool for various applications, it is not intended for real-time, safety-critical systems or for processing sensitive personal information.

Limitations

Users should be aware of the following limitations:

-Biases: Athena-3-7B may exhibit biases present in its training data. Users should critically assess outputs, especially in sensitive contexts.

-Knowledge Cutoff: The model's knowledge is current up to August 2024. It may not be aware of events or developments occurring after this date.

-Language Support: While the model supports multiple languages, performance is strongest in English and technical content.

Acknowledgements

Athena-3-7B builds upon the work of the Qwen team. Gratitude is also extended to the open-source AI community for their contributions to tools and frameworks that facilitated the development of Athena-3-7B.

License

Athena-3-7B is released under the MIT License, permitting wide usage with proper attribution.

---

Use with llama.cpp

Install llama.cpp through brew (works on Mac and Linux)

brew install llama.cpp

Invoke the llama.cpp server or the CLI.

CLI:

llama-cli --hf-repo Triangle104/Athena-3-7B-Q5_K_M-GGUF --hf-file athena-3-7b-q5_k_m.gguf -p "The meaning to life and the universe is"

Server:

llama-server --hf-repo Triangle104/Athena-3-7B-Q5_K_M-GGUF --hf-file athena-3-7b-q5_k_m.gguf -c 2048

Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well.

Step 1: Clone llama.cpp from GitHub.

git clone https://github.com/ggerganov/llama.cpp

Step 2: Move into the llama.cpp folder and build it with LLAMA_CURL=1 flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).

cd llama.cpp && LLAMA_CURL=1 make

Step 3: Run inference through the main binary.

./llama-cli --hf-repo Triangle104/Athena-3-7B-Q5_K_M-GGUF --hf-file athena-3-7b-q5_k_m.gguf -p "The meaning to life and the universe is"

or

./llama-server --hf-repo Triangle104/Athena-3-7B-Q5_K_M-GGUF --hf-file athena-3-7b-q5_k_m.gguf -c 2048