title: Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report
summary: As transformer-based large language models (LLMs) increasingly permeate
  society, they have revolutionized domains such as software engineering,
  creative writing, and digital arts. However, their adoption in cybersecurity
  remains limited due to challenges like scarcity of specialized training data
  and complexity of representing cybersecurity-specific knowledge. To address
  these gaps, we present Foundation-Sec-8B, a cybersecurity-focused LLM built on
  the Llama 3.1 architecture and enhanced through continued pretraining on a
  carefully curated cybersecurity corpus. We evaluate Foundation-Sec-8B across
  both established and new cybersecurity benchmarks, showing that it matches
  Llama 3.1-70B and GPT-4o-mini in certain cybersecurity-specific tasks. By
  releasing our model to the public, we aim to accelerate progress and adoption
  of AI-driven tools in both public and private cybersecurity contexts.
paper_authors._id: 6812f89338bee548818e68a1
paper_authors.name: Paul Kassianik
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-01T04:29:53.088Z
paper_authors._id: 6573a9fe769f3ee9bdf4d9c7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/xC41F7Vp9SVzVHc3cUiRU.jpeg
paper_authors.fullname: Paul Kassianik
paper_authors.type: user
paper_authors.user: paulkass
paper_authors._id: 6812f89338bee548818e68a2
paper_authors.name: Baturay Saglam
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:01:50.509Z
paper_authors._id: 67326fd19417d7f51a9dc559
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/U9a4fHjrJjzXkTFfiafjI.png
paper_authors.fullname: Baturay Saglam
paper_authors.type: user
paper_authors.user: baturaysaglam
paper_authors._id: 6812f89338bee548818e68a3
paper_authors.name: Alexander Chen
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-20T07:23:05.103Z
paper_authors._id: 682bc536a08ed10f5fa177e6
paper_authors.avatarUrl: /avatars/ab5697bb2f806a3955043fa8942ed16a.svg
paper_authors.fullname: Alexander Chen
paper_authors.type: user
paper_authors.user: alexchen92
paper_authors._id: 6812f89338bee548818e68a4
paper_authors.name: Blaine Nelson
paper_authors._id: 6812f89338bee548818e68a5
paper_authors.name: Anu Vellore
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:02:16.802Z
paper_authors._id: 651c8d2909debbe627af9fc5
paper_authors.avatarUrl: /avatars/332c158d0d157c328fbc65e25cbdd02f.svg
paper_authors.fullname: Anu Vellore
paper_authors.type: user
paper_authors.user: anuvellore
paper_authors._id: 6812f89338bee548818e68a6
paper_authors.name: Massimo Aufiero
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:02:22.238Z
paper_authors._id: 636a82a34d1bade3ca8478b8
paper_authors.avatarUrl: /avatars/936b38f82df08d3e71277ff759b85594.svg
paper_authors.fullname: Massimo Aufiero
paper_authors.type: user
paper_authors.user: aufieroma
paper_authors._id: 6812f89338bee548818e68a7
paper_authors.name: Fraser Burch
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-02T06:34:53.958Z
paper_authors._id: 6510b3fabfb7dd98bbf55fa6
paper_authors.avatarUrl: /avatars/a364283cc9deba42500feaa04a84dffc.svg
paper_authors.fullname: Fraser Burch
paper_authors.type: user
paper_authors.user: Frase
paper_authors._id: 6812f89338bee548818e68a8
paper_authors.name: Dhruv Kedia
paper_authors._id: 6812f89338bee548818e68a9
paper_authors.name: Avi Zohary
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:02:44.481Z
paper_authors._id: 67f5bc1288885c9950bc3060
paper_authors.avatarUrl: /avatars/b9bcbdf9fdad008c46ec8085f3415d60.svg
paper_authors.fullname: Avi Zohary
paper_authors.type: user
paper_authors.user: AviZo
paper_authors._id: 6812f89338bee548818e68aa
paper_authors.name: Sajana Weerawardhena
paper_authors._id: 6812f89338bee548818e68ab
paper_authors.name: Aman Priyanshu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-02T06:34:56.066Z
paper_authors._id: 620042b28c2eb991da50d34e
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/620042b28c2eb991da50d34e/Q5cj1GIq3XnKj-K64Mtyd.jpeg
paper_authors.fullname: Aman Priyanshu
paper_authors.type: user
paper_authors.user: AmanPriyanshu
paper_authors._id: 6812f89338bee548818e68ac
paper_authors.name: Adam Swanda
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:03:23.920Z
paper_authors._id: 6388d57b5a3d2a3356240fc0
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1669911914384-noauth.jpeg
paper_authors.fullname: Adam Swanda
paper_authors.type: user
paper_authors.user: deadbits
paper_authors._id: 6812f89338bee548818e68ad
paper_authors.name: Amy Chang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-06-04T09:04:52.140Z
paper_authors._id: 67db21120811453f6e6d1441
paper_authors.avatarUrl: /avatars/0bc82fbf8ce2a199329e875767e33e33.svg
paper_authors.fullname: Amy Chang
paper_authors.type: user
paper_authors.user: n0tjustanychang
paper_authors._id: 6812f89338bee548818e68ae
paper_authors.name: Hyrum Anderson
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:03:30.859Z
paper_authors._id: 62b5e2b5a373ba2c53ebace4
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1656087519480-62b5e2b5a373ba2c53ebace4.jpeg
paper_authors.fullname: Hyrum Anderson
paper_authors.type: user
paper_authors.user: drhyrum
paper_authors._id: 6812f89338bee548818e68af
paper_authors.name: Kojin Oshiba
paper_authors._id: 6812f89338bee548818e68b0
paper_authors.name: Omar Santos
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:04:30.924Z
paper_authors._id: 644f49b2030210812f478a23
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/644f49b2030210812f478a23/Wzee6t75EMu7Z4eGgOMq0.jpeg
paper_authors.fullname: Omar Santos
paper_authors.type: user
paper_authors.user: santosomar
paper_authors._id: 6812f89338bee548818e68b1
paper_authors.name: Yaron Singer
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-05-05T08:04:25.095Z
paper_authors._id: 65028970d4b64cadd2d2ae58
paper_authors.avatarUrl: /avatars/b6b81b853babffa9abfd395c51eaaef1.svg
paper_authors.fullname: Yaron Singer
paper_authors.type: user
paper_authors.user: yaronsinger
paper_authors._id: 6812f89338bee548818e68b2
paper_authors.name: Amin Karbasi
paper_title: Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report
paper_summary: As transformer-based large language models (LLMs) increasingly permeate
  society, they have revolutionized domains such as software engineering,
  creative writing, and digital arts. However, their adoption in cybersecurity
  remains limited due to challenges like scarcity of specialized training data
  and complexity of representing cybersecurity-specific knowledge. To address
  these gaps, we present Foundation-Sec-8B, a cybersecurity-focused LLM built on
  the Llama 3.1 architecture and enhanced through continued pretraining on a
  carefully curated cybersecurity corpus. We evaluate Foundation-Sec-8B across
  both established and new cybersecurity benchmarks, showing that it matches
  Llama 3.1-70B and GPT-4o-mini in certain cybersecurity-specific tasks. By
  releasing our model to the public, we aim to accelerate progress and adoption
  of AI-driven tools in both public and private cybersecurity contexts.
paper_ai_summary: Foundation-Sec-8B, a LLM enhanced with cybersecurity-specific training, matches high-performance models in cybersecurity tasks and promotes AI adoption in the field.
paper_ai_keywords: transformer-based large language models LLMs Llama 3.1 GPT-4o-mini cybersecurity-focused LLM continued pretraining cybersecurity corpus cybersecurity benchmarks