Soft Measures for Extracting Causal Collective Intelligence
<|reference_start|>Soft Measures for Extracting Causal Collective Intelligence: Understanding and modeling collective intelligence is essential for addressing complex social systems. Directed graphs called fuzzy cognitive maps (FCMs) offer a powerful tool for encoding causal mental models, but extracting high-integrity FCMs from text is challenging. This study presents an approach using large language models (LLMs) to automate FCM extraction. We introduce novel graph-based similarity measures and evaluate them by correlating their outputs with human judgments through the Elo rating system. Results show positive correlations with human evaluations, but even the best-performing measure exhibits limitations in capturing FCM nuances. Fine-tuning LLMs improves performance, but existing measures still fall short. This study highlights the need for soft similarity measures tailored to FCM extraction, advancing collective intelligence modeling with NLP.<|reference_end|>
@article{berijanian2024soft,
    title={Soft Measures for Extracting Causal Collective Intelligence},
    author={Maryam Berijanian, Spencer Dork, Kuldeep Singh, Michael Riley
  Millikan, Ashlin Riggs, Aadarsh Swaminathan, Sarah L. Gibbs, Scott E.
  Friedman, Nathan Brugnone},
    journal={arXiv preprint arXiv:2409.18911},
    year={2024},
    archivePrefix={arXiv},
    eprint={2409.18911},
    primaryClass={cs.CL cs.AI cs.CY cs.SI}
}