Decoding the Diversity: A Review of the Indic AI Research Landscape
This review paper provides a comprehensive overview of large language model
(LLM) research directions within Indic languages. Indic languages are those
spoken in the Indian subcontinent, including India, Pakistan, Bangladesh, Sri
Lanka, Nepal, and Bhutan, among others. These languages have a rich cultural
and linguistic heritage and are spoken by over 1.5 billion people worldwide.
With the tremendous market potential and growing demand for natural language
processing (NLP) based applications in diverse languages, generative
applications for Indic languages pose unique challenges and opportunities for
research. Our paper deep dives into the recent advancements in Indic generative
modeling, contributing with a taxonomy of research directions, tabulating 84
recent publications. Research directions surveyed in this paper include LLM
development, fine-tuning existing LLMs, development of corpora, benchmarking
and evaluation, as well as publications around specific techniques, tools, and
applications. We found that researchers across the publications emphasize the
challenges associated with limited data availability, lack of standardization,
and the peculiar linguistic complexities of Indic languages. This work aims to
serve as a valuable resource for researchers and practitioners working in the
field of NLP, particularly those focused on Indic languages, and contributes to
the development of more accurate and efficient LLM applications for these
languages.
666fc2679494795851c7e2d1
Sankalp KJ
admin_assigned
2024-06-17T16:28:48.049Z
664e218ea10258b399d5c358
/avatars/77f2ee0949560504e1643263fd7084da.svg
Sankalp KJ
user
SankalpKJ
666fc2679494795851c7e2d2
Vinija Jain
666fc2679494795851c7e2d3
Sreyoshi Bhaduri
666fc2679494795851c7e2d4
Tamoghna Roy
666fc2679494795851c7e2d5
Aman Chadha
claimed_verified
2024-06-17T07:31:44.640Z
63a4754927f1f64ed7238dac
https://cdn-avatars.huggingface.co/v1/production/uploads/63a4754927f1f64ed7238dac/aH-eJF-31g4vof9jv2gmI.jpeg
Aman Chadha
user
amanchadha
Decoding the Diversity: A Review of the Indic AI Research Landscape
This review paper provides a comprehensive overview of large language model
(LLM) research directions within Indic languages. Indic languages are those
spoken in the Indian subcontinent, including India, Pakistan, Bangladesh, Sri
Lanka, Nepal, and Bhutan, among others. These languages have a rich cultural
and linguistic heritage and are spoken by over 1.5 billion people worldwide.
With the tremendous market potential and growing demand for natural language
processing (NLP) based applications in diverse languages, generative
applications for Indic languages pose unique challenges and opportunities for
research. Our paper deep dives into the recent advancements in Indic generative
modeling, contributing with a taxonomy of research directions, tabulating 84
recent publications. Research directions surveyed in this paper include LLM
development, fine-tuning existing LLMs, development of corpora, benchmarking
and evaluation, as well as publications around specific techniques, tools, and
applications. We found that researchers across the publications emphasize the
challenges associated with limited data availability, lack of standardization,
and the peculiar linguistic complexities of Indic languages. This work aims to
serve as a valuable resource for researchers and practitioners working in the
field of NLP, particularly those focused on Indic languages, and contributes to
the development of more accurate and efficient LLM applications for these
languages.
This paper reviews advances in large language model research for Indic languages, highlighting challenges such as limited data, lack of standardization, and linguistic complexity.
large language models LLM Indic languages generative modeling fine-tuning corpora benchmarking evaluation linguistic complexity