Supernova Event Dataset: Interpreting Large Language Model's Personality
  through Critical Event Analysis
Large Language Models (LLMs) are increasingly integrated into everyday
applications. As their influence grows, understanding their decision making and
underlying personality becomes essential. In this work, we interpret model
personality using our proposed Supernova Event Dataset, a novel dataset with
diverse articles spanning biographies, historical events, news, and scientific
discoveries. We use this dataset to benchmark LLMs on extracting and ranking
key events from text, a subjective and complex challenge that requires
reasoning over long-range context and modeling causal chains. We evaluate small
models like Phi-4, Orca 2, and Qwen 2.5, and large, stronger models such as
Claude 3.7, Gemini 2.5, and OpenAI o3, and propose a framework where another
LLM acts as a judge to infer each model's personality based on its selection
and classification of events. Our analysis shows distinct personality traits:
for instance, Orca 2 demonstrates emotional reasoning focusing on interpersonal
dynamics, while Qwen 2.5 displays a more strategic, analytical style. When
analyzing scientific discovery events, Claude Sonnet 3.7 emphasizes conceptual
framing, Gemini 2.5 Pro prioritizes empirical validation, and o3 favors
step-by-step causal reasoning. This analysis improves model interpretability,
making them user-friendly for a wide range of diverse applications.
6850cb005e07650ecce88fc9
Pranav Agarwal
extracted_confirmed
2025-06-17T01:55:41.273Z
64b89f096c57038f205a7751
/avatars/c268578685cf5b9f0e37ffbdcf239827.svg
Pranav Agarwal
user
pranavAL2109
6850cb005e07650ecce88fca
Ioana Ciucă
Supernova Event Dataset: Interpreting Large Language Model's Personality
  through Critical Event Analysis
Large Language Models (LLMs) are increasingly integrated into everyday
applications. As their influence grows, understanding their decision making and
underlying personality becomes essential. In this work, we interpret model
personality using our proposed Supernova Event Dataset, a novel dataset with
diverse articles spanning biographies, historical events, news, and scientific
discoveries. We use this dataset to benchmark LLMs on extracting and ranking
key events from text, a subjective and complex challenge that requires
reasoning over long-range context and modeling causal chains. We evaluate small
models like Phi-4, Orca 2, and Qwen 2.5, and large, stronger models such as
Claude 3.7, Gemini 2.5, and OpenAI o3, and propose a framework where another
LLM acts as a judge to infer each model's personality based on its selection
and classification of events. Our analysis shows distinct personality traits:
for instance, Orca 2 demonstrates emotional reasoning focusing on interpersonal
dynamics, while Qwen 2.5 displays a more strategic, analytical style. When
analyzing scientific discovery events, Claude Sonnet 3.7 emphasizes conceptual
framing, Gemini 2.5 Pro prioritizes empirical validation, and o3 favors
step-by-step causal reasoning. This analysis improves model interpretability,
making them user-friendly for a wide range of diverse applications.
The study evaluates various LLMs on diverse text tasks using a new dataset, revealing distinct personality traits and improving model interpretability.
Supernova Event Dataset LLMs key event extraction reasoning long-range context causal chains model personality emotional reasoning strategic analytical style conceptual framing empirical validation step-by-step causal reasoning