title: DarkBERT: A Language Model for the Dark Side of the Internet
summary: Recent research has suggested that there are clear differences in the
  language used in the Dark Web compared to that of the Surface Web. As studies
  on the Dark Web commonly require textual analysis of the domain, language
  models specific to the Dark Web may provide valuable insights to researchers.
  In this work, we introduce DarkBERT, a language model pretrained on Dark Web
  data. We describe the steps taken to filter and compile the text data used to
  train DarkBERT to combat the extreme lexical and structural diversity of the
  Dark Web that may be detrimental to building a proper representation of the
  domain. We evaluate DarkBERT and its vanilla counterpart along with other
  widely used language models to validate the benefits that a Dark Web domain
  specific model offers in various use cases. Our evaluations show that DarkBERT
  outperforms current language models and may serve as a valuable resource for
  future research on the Dark Web.
paper_authors._id: 6462f02dd9b4902188122137
paper_authors.name: Youngjin Jin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-05-17T08:04:30.119Z
paper_authors._id: 6200c0ea93ccb1407fe4e57a
paper_authors.avatarUrl: /avatars/af937bea147867c18de387c1e50259e1.svg
paper_authors.fullname: Youngjin Jin
paper_authors.type: user
paper_authors.user: Youngjin
paper_authors._id: 6462f02dd9b4902188122138
paper_authors.name: Eugene Jang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-01-25T08:05:51.394Z
paper_authors._id: 64a3ad72ed01f3dea26bfde6
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64a3ad72ed01f3dea26bfde6/Z7TmrRjUUukKsZ_jRWT2-.png
paper_authors.fullname: Eugene Jang
paper_authors.type: user
paper_authors.user: genesith
paper_authors._id: 6462f02dd9b4902188122139
paper_authors.name: Jian Cui
paper_authors._id: 6462f02dd9b490218812213a
paper_authors.name: Jin-Woo Chung
paper_authors._id: 6462f02dd9b490218812213b
paper_authors.name: Yongjae Lee
paper_authors._id: 6462f02dd9b490218812213c
paper_authors.name: Seungwon Shin
paper_title: DarkBERT: A Language Model for the Dark Side of the Internet
paper_summary: Recent research has suggested that there are clear differences in the
  language used in the Dark Web compared to that of the Surface Web. As studies
  on the Dark Web commonly require textual analysis of the domain, language
  models specific to the Dark Web may provide valuable insights to researchers.
  In this work, we introduce DarkBERT, a language model pretrained on Dark Web
  data. We describe the steps taken to filter and compile the text data used to
  train DarkBERT to combat the extreme lexical and structural diversity of the
  Dark Web that may be detrimental to building a proper representation of the
  domain. We evaluate DarkBERT and its vanilla counterpart along with other
  widely used language models to validate the benefits that a Dark Web domain
  specific model offers in various use cases. Our evaluations show that DarkBERT
  outperforms current language models and may serve as a valuable resource for
  future research on the Dark Web.
paper_ai_keywords: DarkBERT language model pretrained text data lexical diversity structural diversity representation evaluation use cases