MISHANM/google-gemma-2-27b-it.gguf

This model is a GGUF version of the Google gemma-2-27b-it model, optimized for use with the llama.cpp framework. It is designed to run efficiently on CPUs and can be used for various natural language processing tasks.

Model Details

1. Language: English
2. Tasks: Text generation
3. Base Model: google/gemma-2-27b-it

Building and Running the Model

To build and run the model using llama.cpp, follow these steps:

Model

Steps to Download the Model:

1. Go to the "Files and Versions" section.
2. Click on the model.
3. Copy the download link.
4. Create a directory (e.g., for Linux: mkdir gemma2-27b).
5. Navigate to that directory (cd gemma2-27b).
6. Download both model parts: model_part_aa and model_part_ab (e.g., using wget with the copied link).

After downloading the model parts, use the following command to combine them into a complete model:

cat model_part_aa model_part_ab > google-gemma-2-27b-it.gguf

Build llama.cpp Locally

git clone https://github.com/ggerganov/llama.cpp  
cd llama.cpp  
cmake -B build  
cmake --build build --config Release  

Run the Model

Navigate to the build directory and run the model with a prompt:

cd llama.cpp/build/bin   

Inference with llama.cpp

./llama-cli -m /path/to/model/ -p "Your prompt here" -n 128 --ctx-size 8192 --temp 0.6 --seed 3407

Citation Information

@misc{MISHANM/google-gemma-2-27b-it.gguf,
  author = {Mishan Maurya},
  title = {Introducing Google gemma-2-27b-it GGUF Model},
  year = {2025},
  publisher = {Hugging Face},
  journal = {Hugging Face repository},
  
}