title: Evaluate Bias without Manual Test Sets: A Concept Representation
    Perspective for LLMs
summary: Bias in Large Language Models (LLMs) significantly undermines their
  reliability and fairness. We focus on a common form of bias: when two reference
  concepts in the model's concept space, such as sentiment polarities (e.g.,
  "positive" and "negative"), are asymmetrically correlated with a third, target
  concept, such as a reviewing aspect, the model exhibits unintended bias. For
  instance, the understanding of "food" should not skew toward any particular
  sentiment. Existing bias evaluation methods assess behavioral differences of
  LLMs by constructing labeled data for different social groups and measuring
  model responses across them, a process that requires substantial human effort
  and captures only a limited set of social concepts. To overcome these
  limitations, we propose BiasLens, a test-set-free bias analysis framework based
  on the structure of the model's vector space. BiasLens combines Concept
  Activation Vectors (CAVs) with Sparse Autoencoders (SAEs) to extract
  interpretable concept representations, and quantifies bias by measuring the
  variation in representational similarity between the target concept and each of
  the reference concepts. Even without labeled data, BiasLens shows strong
  agreement with traditional bias evaluation metrics (Spearman correlation r >
  0.85). Moreover, BiasLens reveals forms of bias that are difficult to detect
  using existing methods. For example, in simulated clinical scenarios, a
  patient's insurance status can cause the LLM to produce biased diagnostic
  assessments. Overall, BiasLens offers a scalable, interpretable, and efficient
  paradigm for bias discovery, paving the way for improving fairness and
  transparency in LLMs.
paper_authors._id: 682f0391f4060fc097d73b75
paper_authors.name: Lang Gao
paper_authors._id: 682f0391f4060fc097d73b76
paper_authors.name: Kaiyang Wan
paper_authors._id: 682f0391f4060fc097d73b77
paper_authors.name: Wei Liu
paper_authors._id: 682f0391f4060fc097d73b78
paper_authors.name: Chenxi Wang
paper_authors._id: 682f0391f4060fc097d73b79
paper_authors.name: Zirui Song
paper_authors._id: 682f0391f4060fc097d73b7a
paper_authors.name: Zixiang Xu
paper_authors._id: 682f0391f4060fc097d73b7b
paper_authors.name: Yanbo Wang
paper_authors._id: 682f0391f4060fc097d73b7c
paper_authors.name: Veselin Stoyanov
paper_authors._id: 682f0391f4060fc097d73b7d
paper_authors.name: Xiuying Chen
paper_title: Evaluate Bias without Manual Test Sets: A Concept Representation
    Perspective for LLMs
paper_summary: Bias in Large Language Models (LLMs) significantly undermines their
  reliability and fairness. We focus on a common form of bias: when two reference
  concepts in the model's concept space, such as sentiment polarities (e.g.,
  "positive" and "negative"), are asymmetrically correlated with a third, target
  concept, such as a reviewing aspect, the model exhibits unintended bias. For
  instance, the understanding of "food" should not skew toward any particular
  sentiment. Existing bias evaluation methods assess behavioral differences of
  LLMs by constructing labeled data for different social groups and measuring
  model responses across them, a process that requires substantial human effort
  and captures only a limited set of social concepts. To overcome these
  limitations, we propose BiasLens, a test-set-free bias analysis framework based
  on the structure of the model's vector space. BiasLens combines Concept
  Activation Vectors (CAVs) with Sparse Autoencoders (SAEs) to extract
  interpretable concept representations, and quantifies bias by measuring the
  variation in representational similarity between the target concept and each of
  the reference concepts. Even without labeled data, BiasLens shows strong
  agreement with traditional bias evaluation metrics (Spearman correlation r >
  0.85). Moreover, BiasLens reveals forms of bias that are difficult to detect
  using existing methods. For example, in simulated clinical scenarios, a
  patient's insurance status can cause the LLM to produce biased diagnostic
  assessments. Overall, BiasLens offers a scalable, interpretable, and efficient
  paradigm for bias discovery, paving the way for improving fairness and
  transparency in LLMs.
paper_ai_summary: BiasLens is a test-set-free framework that uses concept activation vectors and sparse autoencoders to analyze bias in large language models without labeled data, detecting previously undetected forms of bias.
paper_ai_keywords: Concept Activation Vectors Sparse Autoencoders bias evaluation metrics bias analysis framework representational similarity social concepts