RansenEO: An Encoder-Only SLM with Fine-Tuning for ransomware detection in IoT

This model is a lightweight LoRA adapter trained on network traffic data from IoT environments to detect the presence of ransomware. It performs binary classification—distinguishing between normal and ransomware-related network flows—using a DistilBERT-based architecture and is optimized for deployment in resource-constrained IoT devices.

---

Model Details

• Base Model: distilbert-base-uncased
• Adapter Type: LoRA (via PEFT)
• Target Modules: q_lin, v_lin
• LoRA Rank (r): 8
• Epochs Trained: 5
• Model Type: Binary Classifier (Ransomware Detection)
• Intended Use: Detection of ransomware in IoT environment

---

Training Data

• Dataset: Forge-IIoT (IoTForge Pro)

• Reference:
  Umar P, Mullick S, Das R, Nandi A, Banerjee I. IoTForge Pro. IEEE Internet of Things Journal 2024. doi: 10.21227/c4z1-yc52

• The dataset contains la comprehensive security testbed designed to generate a diverse and extensive intrusion dataset for IIoT environments. The testbed simulates various IIoT scenarios, incorporating network topologies and communication protocols to create realistic attack vectors and normal traffic patterns. The generated dataset, named ForgeIIOT, includes various attack types, such as denial-of-service, man-in-the-middle, ransomware, wildcard abuse and malware-based intrusions

---

Evaluation

The model was evaluated on a held-out test set using the following metrics:

• Accuracy
• F1-Score
• Matthews Correlation Coefficient (MCC)
• Inference Time
• Model Size (MB)
• RAM Usage (MB)

These metrics ensure the model is not only accurate but also efficient for use in IoT scenarios.

---

Usage

To use the adapter with the base model in your project:

from transformers import AutoTokenizer, AutoModelForSequenceClassification
from peft import PeftModel, PeftConfig

peft_model_id = "yeico/RansenEO"

# Load PEFT config
config = PeftConfig.from_pretrained(peft_model_id)

# Load base model
base_model = AutoModelForSequenceClassification.from_pretrained(config.base_model_name_or_path)

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, peft_model_id)

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)