ELISARCyberAIEdge7B

Maintainer: Dr. Sabri Sallani Expertise: AI Research & Cybersecurity Adapter type: LoRA (Low-Rank Adaptation) Base model: mistralai/Mistral-7B-v0.1 (FP16) Intended use: Offline edge deployment for CyberAI & Blue-Team scenarios License: Apache 2.0 (see LICENSE)

---

📖 Overview

ELISARCyberAIEdge7B is a LoRA adapter crafted by Dr. Sabri Sallani—AI & cybersecurity researcher—to specialize Mistral-7B for offline, on-device CyberAI and “Blue AI” (defensive) applications. Once merged with the FP16 base, you obtain a single ~5 GB GGUF that runs natively on edge hardware (e.g., Raspberry Pi 4, Jetson Nano, NVIDIA T4) without internet access.

Key points:

• 🔧 LoRA-only: Contains low-rank delta-weights for Mistral-7B.
• 🛠️ Edge-optimized: Full merged GGUF runs entirely offline on typical edge GPUs/accelerators.
• 🚀 Cybersecurity focus: Fine-tuned on “ELISAR CyberAI Edge” corpus—vulnerability descriptions, incident reports, secure-coding examples, threat intelligence summaries.
• 👤 Authored by Dr. Sabri Sallani: Published under the ELISAR initiative.

---

⚙️ Installation

1. Python dependencies

   pip install transformers peft accelerate sentencepiece torch
   

2. (Optional) llama.cpp + GGUF tools (to merge and run offline)

   # Clone and install gguf-py
   git clone --depth 1 https://github.com/ggml-org/llama.cpp.git
   pip install ./llama.cpp/gguf-py
   pip install llama-cpp-python
   

   → Use these tools to merge LoRA + base weights into a single GGUF.

---

🐍 Usage

1. Inference with transformers + PEFT (online GPU/CPU)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

BASE_ID    = "mistralai/Mistral-7B-v0.1"
ADAPTER_ID = "sallani/ELISARCyberAIEdge7B"

# 1) Load Mistral-7B base (FP16 or BF16) with automatic device placement
tokenizer = AutoTokenizer.from_pretrained(BASE_ID, use_fast=True)
base_model = AutoModelForCausalLM.from_pretrained(
    BASE_ID,
    torch_dtype="auto",
    device_map="auto"
)

# 2) Load LoRA adapter on top
model = PeftModel.from_pretrained(
    base_model,
    ADAPTER_ID,
    torch_dtype="auto",
    device_map="auto"
)
model.eval()

# 3) Perform inference
prompt = (
    "### Instruction:\n"
    "Propose a set of secure-coding guidelines for Python web applications.\n"
    "### Response:\n"
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(
        **inputs,
        max_new_tokens=128,
        temperature=0.8,
        top_p=0.9,
        repetition_penalty=1.1
    )

print(tokenizer.decode(out[0], skip_special_tokens=True))

• device_map="auto" places weights on GPU/CPU automatically (FP16 when supported).
• Adjust sampling parameters (temperature, top_p, repetition_penalty) for your use case.

2. Offline Edge Deployment via llama.cpp (Merged GGUF)

1. Merge LoRA + base into a single GGUF

   python3 llama.cpp/convert_lora_to_gguf.py \
     sallani/ELISARCyberAIEdge7B \                # LoRA repo or local folder
     --base-model-id mistralai/Mistral-7B-v0.1 \  # HF ID of FP16 base
     --outfile elisar_full_f16.gguf               # Output GGUF (~5 GB)
   

   • The script pulls the FP16 base automatically from HF, applies LoRA deltas, and writes a merged GGUF.

2. Run inference on edge

   • Copy elisar_full_f16.gguf to your edge device (Jetson Nano, Raspberry Pi 4 + GPU, NVIDIA T4).

   • Use llama.cpp binary to run:

     ./llama.cpp/main \
       -m elisar_full_f16.gguf \
       -p "### Instruction: Audit the following log entries for suspicious activity.\n---\n<log lines>\n---\n### Response:" \
       --temp 0.7 \
       --repeat_penalty 1.1 \
       --n 128
     

   • No internet is required once the GGUF is on-device.

---

📐 Model Details

• Base architecture: Mistral-7B-v0.1 (40 transformer layers, 4096-dim embedding, 32 heads, causal LM).

• LoRA configuration:

  • Rank = 64, α = 16
  • Applied to Q/K/V and feed-forward projections
  • Adapter snapshots ≈ 168 MB

• Training corpus (ELISAR CyberAI Edge):

  • Public vulnerability databases (CVE entries, CVSS scoring).
  • Real-world incident reports (MITRE ATT&CK red vs. blue logs).
  • Secure-coding patterns (OWASP Top 10, SAST examples).
  • Blue-team playbooks and defensive strategies.

• Hyperparameters:

  • Learning rate = 1e-4, batch size = 16 per GPU, 3 epochs on 8×A100 (FP16).
  • Validation on unseen CVE descriptions and red-team prompts.

• Merged GGUF (FP16):

  • ~5 GB total after merging and trimming unnecessary metadata for on-device use.

---

🔖 Prompt Guidelines

• Structured prompt

  ### Instruction:
  <clear cybersecurity or defensive AI task>
  ### Response:
  

• Recommended sampling

  • temperature=0.7–0.9 for balanced creativity.
  • top_p=0.9 for nucleus sampling.
  • repetition_penalty=1.1 to reduce loops.

---

---

language: en license: apache-2.0 tags:

• gguf
• quantized
• cybersecurity
• edge-llm
• lora
• mistral
• elisar model_name: ELISARCyberAIEdge7B-LoRA-GGUF pipeline_tag: text-generation datasets:
• custom widget:
• text: "What are the main threats targeting OT environments?"

📊 Stats & Adoption

• 🔄 Download tracking: Enabled
• 📥 Total downloads (last 30 days): auto-updated by HF
• 🧪 Being tested on:
  • Jetson Nano (Ubuntu 20.04, CUDA 11.4)
  • Raspberry Pi 4 (with Coral TPU)
  • NVIDIA T4 + LLaMA.cpp

Want to share your benchmarks? Open an Issue or pull request.

⚠️ License & Citation

• License: Apache 2.0 (see LICENSE).

• Attribution:

  Sallani, S. (2025). ELISARCyberAIEdge7B: LoRA adapter for Mistral-7B specializing in offline CyberAI Edge tasks. Hugging Face Model Hub: sallani/ELISARCyberAIEdge7B.

---

🛠️ Support & Contact

• Report issues or feature requests: https://huggingface.co/sallani/ELISARCyberAIEdge7B/issues

• Contact the author: Dr. Sabri Sallani • GitHub: @sallani • Email: [email protected] • LinkedIn: linkedin.com/in/sabri-sallani

Thank you for using ELISARCyberAIEdge7B. This adapter empowers secure, offline AI at the edge for next-gen CyberAI and Blue-Team applications.