ﬁne-tuning, and system design. Additionally, the survey introduces a nuanced cat-
egorization of resource eﬃciency techniques by their speciﬁc resource types, which
uncovers the intricate relationships and mappings between various resources and
corresponding optimization techniques. A standardized set of evaluation met-
rics and datasets is also presented to facilitate consistent and fair comparisons
across diﬀerent models and techniques. By oﬀering a comprehensive overview of
the current state-of-the-art and identifying open research avenues, this survey
serves as a foundational reference for researchers and practitioners, aiding them
in developing more sustainable and eﬃcient LLMs in a rapidly evolving land-
scape. To make the ﬁeld more accessible to interested beginners, we not only
make a systematic review of existing works and a highly structured taxonomy of
resource-eﬃcient LLMs but also release a website including a constantly-updated
Beyond Efficiency