LLAMA-3.1-8B-Construction (Merged)

This is a fine-tuned version of Meta's Llama-3.1-8B model optimized for construction industry and building regulations knowledge. This repository contains the full merged model (base + fine-tuning), making it directly usable with the Hugging Face API.

Model Details

Base Model: meta-llama/Meta-Llama-3.1-8B
Fine-tuning Method: LoRA (Low-Rank Adaptation), merged with base model
Training Data: Custom dataset focusing on construction industry standards, building regulations, and safety requirements
Usage: This model is designed to answer questions about building codes, construction best practices, and regulatory compliance

Example Usage

You can use this model directly with the Hugging Face transformers library:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model_id = "SamuelJaja/llama-3.1-8b-construction-merged"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Generate text
prompt = "[INST] What are the main requirements for fire safety in commercial buildings? [/INST]"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    input_ids=inputs.input_ids,
    attention_mask=inputs.attention_mask,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

API Usage

Because this is a merged model (not just an adapter), you can use it directly with the Hugging Face Inference API.

Limitations

This model is specialized for construction knowledge and may not perform as well on general topics outside its domain of specialization.

Citation and Contact

If you use this model in your projects or research, please reference or link to this model card.

For questions or support, please contact me through the Hugging Face community tab.