Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations
<|reference_start|>Detectors for Safe and Reliable LLMs: Implementations, Uses, and Limitations: Large language models (LLMs) are susceptible to a variety of risks, from non-faithful output to biased and toxic generations. Due to several limiting factors surrounding LLMs (training cost, API access, data availability, etc.), it may not always be feasible to impose direct safety constraints on a deployed model. Therefore, an efficient and reliable alternative is required. To this end, we present our ongoing efforts to create and deploy a library of detectors: compact and easy-to-build classification models that provide labels for various harms. In addition to the detectors themselves, we discuss a wide range of uses for these detector models - from acting as guardrails to enabling effective AI governance. We also deep dive into inherent challenges in their development and discuss future work aimed at making the detectors more reliable and broadening their scope.<|reference_end|>
@article{achintalwar2024detectors,
    title={Detectors for Safe and Reliable LLMs: Implementations, Uses, and
  Limitations},
    author={Swapnaja Achintalwar, Adriana Alvarado Garcia, Ateret Anaby-Tavor,
  Ioana Baldini, Sara E. Berger, Bishwaranjan Bhattacharjee, Djallel
  Bouneffouf, Subhajit Chaudhury, Pin-Yu Chen, Lamogha Chiazor, Elizabeth M.
  Daly, Kirushikesh DB, Rog'erio Abreu de Paula, Pierre Dognin, Eitan Farchi,
  Soumya Ghosh, Michael Hind, Raya Horesh, George Kour, Ja Young Lee, Nishtha
  Madaan, Sameep Mehta, Erik Miehling, Keerthiram Murugesan, Manish Nagireddy,
  Inkit Padhi, David Piorkowski, Ambrish Rawat, Orna Raz, Prasanna Sattigeri,
  Hendrik Strobelt, Sarathkrishna Swaminathan, Christoph Tillmann, Aashka
  Trivedi, Kush R. Varshney, Dennis Wei, Shalisha Witherspooon, Marcel
  Zalmanovici},
    journal={arXiv preprint arXiv:2403.06009},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.06009},
    primaryClass={cs.LG}
}