Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report
AmanPriyanshu
Abstract
As transformer-based large language models (LLMs) increasingly permeate society, they have revolutionized domains such as software engineering, creative writing, and digital arts. However, their adoption in cybersecurity remains limited due to challenges like scarcity of specialized training data and complexity of representing cybersecurity-specific knowledge. To address these gaps, we present Foundation-Sec-8B, a cybersecurity-focused LLM built on the Llama 3.1 architecture and enhanced through continued pretraining on a carefully curated cybersecurity corpus. We evaluate Foundation-Sec-8B across both established and new cybersecurity benchmarks, showing that it matches Llama 3.1-70B and GPT-4o-mini in certain cybersecurity-specific tasks. By releasing our model to the public, we aim to accelerate progress and adoption of AI-driven tools in both public and private cybersecurity contexts.
Community
This paper introduces Foundation-Sec-8B, a cybersecurity-focused LLM based on Llama 3.1 architecture with continued pretraining on a specialized security corpus. Evaluation demonstrates comparable performance to larger models on security-specific tasks. The model is a publicly released open-weights model to support more AI adoption within cybersecurity contexts (https://huggingface.co/fdtn-ai/Foundation-Sec-8B).
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- CyberLLMInstruct: A New Dataset for Analysing Safety of Fine-Tuned LLMs Using Cyber Security Data (2025)
- The Digital Cybersecurity Expert: How Far Have We Come? (2025)
- AttackSeqBench: Benchmarking Large Language Models' Understanding of Sequential Patterns in Cyber Attacks (2025)
- Exploring the Role of Large Language Models in Cybersecurity: A Systematic Survey (2025)
- LLM-Assisted Proactive Threat Intelligence for Automated Reasoning (2025)
- ELTEX: A Framework for Domain-Driven Synthetic Data Generation (2025)
- Secret Breach Detection in Source Code with Large Language Models (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 2
Datasets citing this paper 0
No dataset linking this paper