Peacemann/google_gemma-2-9b-it_LMUL

L-Mul Optimized: google/gemma-2-9b-it

This is a modified version of Google's gemma-2-9b-it model. The modification consists of replacing the standard attention mechanism with one that uses a custom, approximate matrix multiplication algorithm termed "L-Mul".

This work was performed as part of a research project to evaluate the performance and accuracy trade-offs of algorithmic substitutions in transformer architectures.

This model is intended strictly for educational and scientific purposes.

Model Description

The core architecture of google/gemma-2-9b-it is preserved. However, the standard Gemma2Attention modules have been dynamically replaced with a custom version that utilizes the l_mul_attention function for its core computations. This function is defined in the lmul.py file included in this repository.

• Base Model: google/gemma-2-9b-it
• Modification: Replacement of standard attention with L-Mul approximate attention.
• Primary Use-Case: Research and educational analysis of algorithmic impact on LLMs.

How to Get Started

To use this model, you must use the trust_remote_code=True flag when loading it. This is required to execute the custom lmul.py file that defines the new attention mechanism.

You can load the model directly from this repository using the transformers library:

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Define the repository ID for the specific model
repo_id = "Peacemann/google_gemma-2-9b-it-lmul-attention" # Replace with the correct repo ID if different

# Load the tokenizer and model, trusting the remote code to load lmul.py
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

# Example usage
prompt = "The L-Mul algorithm is an experimental method for..."
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=50)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Uses & Limitations

This model is intended for researchers and students exploring the internal workings of LLMs. It is a tool for visualizing and analyzing the effects of fundamental algorithmic changes.

This model is NOT intended for any commercial or production application.

The modification is experimental. The impact on the model's performance, safety alignment, accuracy, and potential for generating biased or harmful content is unknown and untested.

Licensing Information

The use of this model is subject to the original Gemma 2 Community License. By using this model, you agree to the terms outlined in the license.