NLP research is impeded by a lack of resources and awareness of the challenges presented by underrepresented languages and dialects. Focusing on the languages spoken in Indonesia, the second most linguistically diverse and the fourth most populous nation of the world, we provide an overview of the current state of NLP research for Indonesia's 700+ languages. We highlight challenges in Indonesian NLP and how these affect the performance of current NLP systems. Finally, we provide general recommendations to help develop NLP technology not only for languages of Indonesia but also other underrepresented languages. Research in natural language processing (NLP) has traditionally focused on developing models for English and a small set of other languages with large amounts of data (see Figure 1, bottom right). While the lack of data is generally cited as the key reason for the lack of progress in NLP for underrepresented languages (Hu et al., 2020; Joshi et al., 2020), we argue that another factor relates to the diversity and the lack of understanding of the linguistic characteristics of such languages. Through the lens of the languages spoken in Indonesia, the world's second-most linguistically diverse country, we seek to illustrate the challenges in applying NLP technology to such a diverse pool of languages. Indonesia is the 4th most populous nation globally, with 273 million people spread over 17,508 islands. There are more than 700 languages spoken in Indonesia, equal to 10% of the world's languages, second only to Papua New Guinea (Eber-hard et al., 2021). However, most of these languages are not well documented in the literature; many are not formally taught, and no established standard exists across speakers (Novitasari et al.,  These authors contributed equally. 2020). Many of them are decreasing in use, as Indonesian ( Bahasa Indonesia ), the national language, is more frequently used as the primary language across the country. This process may ultimately result in a monolingual society (Cohn and Ravindranath, 2014). Among more than 700 Indonesian local languages, many are threatened. 440 languages are listed as endangered and 12 as extinct according to data from Ethnologue (Eberhard et al., 2021) illustrated in Figure 2. Anindyatri and Mufidah (2020) found nearly half of a sample of 98 Indonesian local languages to be endangered while van Esch et al. (2022) observed 71 among 151 Indonesian local languages to have less than 100k speakers. Table 1 lists the names of the 10 most spoken local languages in Indonesia (Eberhard et al., 2021). Javanese and Sundanese are at the top with 84M and 34M speakers, respectively, while Madura, Minangkabau, and Buginese each have around 6M speakers. Despite their large speaker populations, these local languages are poorly represented in the NLP literature. Compared to Indonesian, the number of research papers mentioning these languages has barely increased over the past 20 years (Figure 1, top). Furthermore, compared to their European counterparts, Indonesian languages are drastically understudied (Figure 1, bottom). This is true even for Indonesian, which has nearly 200M speakers. Language technology should be accessible to everyone in their native languages (European Language Resources Association, 2019), including Indonesians. In the context of Indonesia, language technology research offers some benefits. First, language technology is a potential peacemaker tools in a multi-ethnic country, helping Indonesians understand each other better and avoid the ethnic con-flicts of the past (Bertrand, 2004). On a larger scale, language technology promotes language use (Euro-pean Language Resources Association, 2019) and helps language preservation. Despite these benefits, following Bird (2020), we recommend a careful assessment of individual usage scenarios of language technology, so they are implemented for the good of the local population. For language technology to be useful in the Indonesian context, it additionally has to account for the dialects of local languages. Language dialects in Indonesia are influenced by the geographical location and regional culture of their speakers (Van-der Klok, 2015) and thus often differ substantially in morphology and vocabulary, posing challenges for NLP systems. In this paper, we provide an overview of the current state of NLP for Indonesian and Indonesia's hundreds of languages. We then discuss the challenges presented by those languages and demonstrate how they affect state-of-the-art systems in NLP. We finally provide recommendations for developing better NLP technology not only for the languages in Indonesia but also for other underrepresented languages. Indonesia is one of the richest countries globally in terms of linguistic diversity. More than 400 of its languages belong to the Austronesian language family, while the others are Papuan languages spoken in the eastern part of the country. As shown in Figure 3, the Austronesian languages in Indonesia belong to three main groups: Western-Malayo-Polynesian (WMP), Central-Malayo-Polynesian (CMP), and South-Halmahera-West-New-Guinea (SHWNG) (Blust, 1980). WMP languages are Malay, Indonesian, Javanese, Sundanese, Balinese, and Minangkabau, among others. All languages mentioned in Table 1 are in this group. Lan-7227 Figure 3: Map of Austronesian and Papuan languages in Indonesia. guages belonging to CMP are languages of the Lesser Sunda Islands from East Sumbawa (with Bimanese) onwards to the east, and languages of the central and southern Moluccas (including the Aru Islands and the Sula Archipelago). The SHWNG group consists of languages of Halmahera and Cenderawasih Bay, and further-flung regions such as the Mamberamo River and the Raja Ampat Islands. Meanwhile, the Papuan languages are mainly spoken in Papua, such as Dani, Asmat, Maybrat, and Sentani. Some Papuan languages are also spoken in Halmahera, Timor, and the Alor Archipelago (Palmer, 2018; Ross, 2005). Most Austronesian linguists and archaeologists agree that the original homeland' of Austronesian languages must be sought in Taiwan and, prior to Taiwan, in coastal South China (Adelaar, 2005; Bellwood et al., 2011). In the second millennium CE, the Austronesian people moved from Taiwan to the Philippines. From the Philippines, they moved southward to Borneo and Sulawesi. From Borneo, they migrated to Sumatra, the Malay Peninsula, Java, and even to Madagascar. From Sulawesi, they moved southward to the CMP area and eastward to the SHWNG area. From there, they migrated to Oceania and Polynesia, as far as New Zealand, Easter Island, and Hawaii (Gray and Jordan, 2000). The people that lived in insular Southeast Asia, such as in the Philippines and Indonesia, before the arrival of Austronesians were Australo-Melanesians (Bellwood, 1997). Gradual assimilation with Austronesians occurred, although some pre-Austronesian groups still survive, such as Melanesian people in eastern Indonesia (Ross, 2005; Coupe and Kratochvl, 2020). franca) of interethnic communication in Southeast Asia and beyond (Steinhauer, 2005; Coupe and Kratochvl, 2020). It functioned as the language of trade and the language of Islam because Muslim merchants from India and the Middle East were the first to introduce the religion into the harbor towns of Indonesia. After the arrival of Europeans, Malay was used by the Portuguese and Dutch to spread Catholicism and Protestantism. When the Dutch extended their rule over areas outside Java in the nineteenth century, the importance of Malay increased, and thus, the first standardization of the spelling and grammar occurred in 1901, based on Classical Malay (Abas, 1987; Sneddon, 2003). In 1928, the Second National Youth Congress participants proclaimed Malay (henceforth called Indonesian) as the unifying language of Indonesia. During World War II, the Japanese occupying forces forbade all use of Dutch in favor of Indonesian, which from then onward effectively became the new national language. From independence until the present, Indonesian has functioned as the primary language in education, mass media, and government activities. Many local language speakers are increasingly using Indonesian with their children because they believe it will aid them to attain a better education and career (Klamer, 2018). Recently, pretrained multilingual language models such as mBERT (Devlin et al., 2019), mBART (Liu et al., 2020), and mT5 (Xue et al., 2021b) have been proposed. Their coverage, however, focuses on high-resource languages. Only mBERT and mT5 include Indonesian local languages, i.e., Javanese, Sundanese, and Minangkabau, but with comparatively little pretraining data. Some multilingual datasets for question answering (TyDiQA; Clark et al., 2020), common sense reasoning (XCOPA; Ponti et al., 2020), abstractive summarization (Hasan et al., 2021), passage ranking (mMARCO; Bonifacio et al., 2021), cross-lingual visual question answering (xGQA; Pfeif-fer et al., 2021), language and vision reasoning (MaRVL; Liu et al., 2021), paraphrasing (Para-Cotta; Aji et al., 2021), dialogue systems (XPer-sona & BiToD; Lin et al., 2021a,b), lexical normalization (MultiLexNorm; van der Goot et al., 2021), and machine translation (FLORES-101; Guzmn et al., 2019) include Indonesian but most others do not, and very few include Indonesian local lan-7228 guages. An exception is the weakly supervised named entity recognition dataset, WikiAnn (Pan et al., 2017), which covers several Indonesian local languages, namely Acehnese, Javanese, Minangkabau, and Sundanese. Parallel corpora including Indonesian local languages are: (i) CommonCrawl; (ii) Wikipedia parallel corpora like MediaWiki Translations; 1 and WikiMatrix (Schwenk et al., 2021) (iii) the Leipzig corpora (Goldhahn et al., 2012), which include Indonesian, Javanese, Sundanese, Minangkabau, Madurese, Acehnese, Buginese, Banjar, and Balinese; and (iv) JW-300 (Agic and Vulic, 2019), which includes dozens of Indonesian local languages, e.g., Batak language groups, Javanese, Dayak language groups, and several languages in Nusa Tenggara. Recent studies, however, have raised concerns regarding the quality of such multilingual corpora for underrepresented languages (Caswell et al., 2022). NLP research on Indonesian has occurred across multiple topics, such as POS tagging (Wicaksono and Purwarianti, 2010; Dinakaramani et al., 2014), NER (Budi et al., 2005; Rachman et al., 2017; Gunawan et al., 2018), sentiment analysis (Narad-hipa and Purwarianti, 2011; Lunando and Purwarianti, 2013; Wicaksono et al., 2014), hate speech detection (Alfina et al., 2017; Sutejo and Lestari, 2018), topic classification (Winata and Khodra, 2015; Kusumaningrum et al., 2016), question answering (Mahendra et al., 2008; Fikri and Purwarianti, 2012), machine translation (Yulianti et al., 2011; Simon and Purwarianti, 2013; Hermanto et al., 2015), keyphrases extraction (Saputra et al., 2018; Trisna and Nurwidyantoro, 2020), morphological analysis (Pisceldo et al., 2008), and speech recognition (Lestari et al., 2006; Baskoro and Adriani, 2008; Zahra et al., 2009). However, many of these studies either did not release the data or used non-standardized resources with a lack of documentation and open source code, making them extremely difficult to reproduce. Recently, Wilie et al. (2020), Koto et al. (2020b, 2021), and Cahyawijaya et al. (2021) collected Indonesian NLP resources as benchmark data. Others have also begun to create standardized labeled data for Indonesian NLP, e.g. the works of Kurniawan and Aji (2018), Guntara et al. (2020), Koto et al. 1 https://mediawiki.org/wiki/Content_translation (2020a), Khairunnisa et al. (2020), and Mahendra et al. (2021). On the other hand, there has been very little work on local languages. Several works studied stemming (Sundanese (Suryani et al., 2018); Balinese (Subali and Fatichah, 2019)) and POS Tagging (Madurese; Dewi et al., 2020). Koto and Koto (2020) built a Indonesian Minangkabau parallel corpus and also sentiment analysis resources for Minangkabau. Other works developed machine translation systems between Indonesian and local languages, e.g., Sundanese (Suryani et al., 2015), Buginese (Apriani et al., 2016), Dayak Kanayatn (Hasbiansyah et al., 2016), and Sambas Malay (Ningtyas et al., 2018). Tanaya and Adriani (2016, 2018) studied Javanese character segmentation in non-Latin script. Safitri et al. (2016) worked on spoken data language identification in Minangkabau, Sundanese, and Javanese, while Azizah et al. (2020) developed end-to-end neural text-to-speech models for Indonesian, Sundanese, and Javanese. Nasution et al. (2017, 2021) proposed an approach for bilingual lexicon induction and evaluated the approach on seven languages, i.e., Indonesian, Malay, Minangkabau, Palembang Malay, Banjar, Javanese, and Sundanese. Cahyawijaya et al. (2021) established a machine translation benchmark in Sundanese and Javanese using Bible data. Wibowo et al. (2021) studied a family of colloquial Indonesian, which is influenced by some local languages via morphological transformation, and Putri et al. (2021) worked on abusive language and hate speech detection on Twitter for five local languages, namely Javanese, Sundanese, Madurese, Minangkabau, and Musi. Monolingual Data Unlabeled corpora are crucial for building large language models, such as GPT-2 (Radford et al., 2019) or BERT (Devlin et al., 2019). Available unlabeled corpora such as Indo4B (Wilie et al., 2020), and Indo4B-Plus (Cahyawijaya et al., 2021) mainly include data in Indonesian, with the latter containing  10% of data in Javanese and Sundanese. In comparison, in multilingual corpora such as CC100 (Conneau et al., 2020), Javanese and Sundanese data accounts for only 0.001% and 0.002% of the corpus size respectively while in mC4 (Xue 7229 # speakers (Million) W i k i ped i a s i z e ( GB ) 0.0001 0.001 0.01 0.1 1 10 0.001 0.01 0.1 1 10 100 1000 Europe Asia Indonesia Figure 4: Relationship between the number of speakers and the size of data in Wikipedia for languages spoken in Europe, Asia, and Indonesia. et al., 2021b), there are only 0.6M Javanese and 0.3M Sundanese tokens out of a total of 6.3T tokens. In addition, we measure data availability in Wikipedia compared to the number of speakers in Figure 4. 2 Much less data is available for the languages spoken in Indonesia, compared to European languages with similar numbers of speakers. For example, Wikipedia contains more than 3 GB of Italian articles but less than 50 MB of Javanese articles, despite both languages having a comparable number of speakers. Similarly, Sundanese has less than 25 MB of articles, whereas languages with comparable numbers of speakers have more than 1.5 GB of articles. Similar trends hold for most other Asian languages. Languages in Africa are even more underrepresented in terms of Wikipedia data (see Appendix B). Beyond the highly spoken local languages, most other Indonesian local languages do not have Wikipedia instances, in contrast to European languages with few speakers. It is very difficult to find alternative sources for high-quality text data for other local languages of Indonesia (such as news websites), as most such sources are written in Indonesian. Resources in long-tail languages are even more scarce due to a very low number of speakers. Moreover, most of the languages in the long tail are mainly used in a spoken context, making text data challenging to obtain. These statistics demonstrate 2 The number of speakers is collected from Wiki-data (Vrandecic and Krtzsch, 2014), from the number of speakers (P1098) property as of Nov 7th, 2021, while the size is collected from the 20211101 Wikipedia dump. that collecting unlabeled corpora for Indonesian local languages is extremely difficult. This makes it impractical to develop strong pretrained language models for these languages, which have been the foundation for many recent NLP systems. Labeled Data Most work on Indonesian NLP (see 2) has not publicly released the data or models, limiting reproducibility. Although recent Indonesian NLP benchmarks are addressing this issue, they mostly focus on the Indonesian language (see Appendix F). Some widely spoken local languages such as Javanese, Sundanese, or Minangkabau have extremely small labeled datasets compared to Indonesian, while others have barely any. The lack of such datasets makes NLP development for the local languages difficult. However, constructing new labeled datasets is still challenging due to: (1) the lack of speakers of some languages; (2) the vast continuum of dialectical variation (see 3.2.1); and (3) the absence of writing standard in most local languages (see 3.3). The diversity of Indonesian languages is not only reflected in the large number of local languages but also the large number of dialects of these languages (3.2.1). Speakers of local languages also often mix languages in conversation, which makes colloquial Indonesian more diverse (3.2.2). In addition, some local languages are more commonly used in conversational contexts, so they do not have consistent writing forms in written media (3.3). Indonesian local languages often have multiple dialects, depending on the geographical location. Local languages of Indonesian spoken in different locations might be different (have some lexical variation) to one another, despite still being categorized as the same language (Fauzi and Puspitorini, 2018). For example, Anderbeck (2008) showed that villages across the Jambi province use different dialects of Jambi Malay. Similarly, Kartikasari et al. (2018) showed that Javanese between different cities in central and eastern Java could have more than 50% lexical variation, while Purwan-ingsih (2017) showed that Javanese in different districts in the Lamongan has up to 13% lexical variation. Similar studies have been conducted on other languages, such as Balinese (Maharani and Candra, 2018) and Sasak (Sarwadi et al., 2019). Moreover, Indonesian and its local languages have multiple styles, even within the same dialect. One factor that affects style is the level of politeness and formalitysimilar to Japanese and other Asian languages (Bond and Baldwin, 2016). More polite language is used when speaking to a person with a higher social position, especially to elders, seniors, and sometimes strangers. Different politeness levels manifest in the use of different honorifics and even different lexical terms. To illustrate the distinctions between regional dialects and styles, we highlight common words and utterances across dialects and styles in Jambi Malay and Javanese in Tables 2 and 3 respectively. For Jambi Malay, we sample the result from a prior work (Anderbeck, 2008). For Javanese, we ask native speakers to translate basic words into three regional dialects: Western, Central, and Eastern Javanese, and two different styles: Ngoko (stan-dard, daily-use Javanese) and Krama (polite Javanese, used to communicate to elders and those with higher social status). However, since contemporary Krama Javanese is not very different among regions, we only consider Krama from the Eastern speakers' perspective. Jambi Malay has many dialects across villages. As shown in Table 2, many common words are spoken differently across dialects and styles. Similarly, Javanese is also different across regions. Not every Javanese speaker understands Krama , since its usage is very limited. Moreover, the number of Javanese speakers who can use Krama is declin-Model Style Region langid.py FastText CLD3 Top-1 Top-3 Top-1 Top-3 Top-1 Ngoko Western 0.241 0.621 0.069 0.379 0.759 Ngoko Central 0.345 0.690 0.379 0.724 0.828 Ngoko Eastern 0.276 0.552 0.103 0.379 0.552 Krama Eastern 0.345 0.759 0.379 0.586 0.897 Table 4: Language identification accuracy based on different Javanese dialects and styles. Dialectical and style differences pose a challenge to NLP systems. To explore the extent of this challenge, we conduct an experiment to test the robustness of NLP systems to variations in Javanese dialects. We ask native speakers 4 to translate 29 simple sentences into Javanese according to the specified dialect and style. We then evaluate several language identification systems on those instances. Language identification is a core part of multilingual NLP and a necessary step for collect-3 Krama is used to speak formally (e.g., with older or respected people). However, people prefer to use Indonesian more in a formal situation. People who move from sub-urban areas to bigger cities tend to continue to use Ngoko and thus also pass Ngoko on to their children. 4 Our annotators are based in Banyumas, Jogjakarta, and Jember for Western, Central, and Eastern Javanese respectively. Using dialects from different cities might yield different results. ing textual data in a language. Despite its importance, it is an open research area, particularly for underrepresented languages (Hughes et al., 2006, Caswell et al., 2022). We compare langid.py (Lui and Baldwin, 2012), FastText (Joulin et al., 2017), and CLD3. 5 The results can be seen in Table 4. In general, the language identification systems are more accurate in detecting Javanese texts in the Ngoko -Central dialect, or Krama , since the systems were trained on Javanese Wikipedia data, which is written in either the Ngoko -Central or Krama dialects and styles. If an NLP system can only detect certain dialects, then this information should be conveyed explicitly. Problems arise if we assume that the model works equally well across dialects. For example, in the case of language identification, if we use the model to collect datasets automatically, then Javanese datasets with poor-performing dialects will be underrepresented in the data. Code-mixing is an occurrence where a person speaks alternately in two or more languages in a conversation (Sitaram et al., 2019, Winata et al., 2018, 2019b, Dogruz et al., 2021). This phenomenon is common in Indonesian conversations (Barik et al., 2019, Johanes et al., 2020, Wibowo et al., 2021). In a conversational context, people sometimes mix their local languages with standard Indonesian, resulting in colloquial Indonesian (Siregar et al., 2014). This colloquial-style Indonesian is used daily in speech and conversation and is common on social media (Sutrisno and Ariesta, 2019). Some frequently used code-mixed words (especially on social media) are even intelligible to people that do not speak the original local languages. Interestingly, code-mixing can also occur in border areas where people are exposed to 5 https://github.com/google/cld3 Language Meaning Written Variation IPA Javanese what apa / opo /OpO/ (Eastern there is ana / ono / onok /OnOP/ Ngoko ) you kon / koen /kOn/ Balinese yes inggih / nggih /PNgih/ (Alus I / me tiang / tyang /tiaN/ Singgih ) <greeting> swastyastu / swastiastu /swastiastu/ Sundanese please / sorry punten / punteun /punt@n/ (Badui red beureum / berem /b@r1m/ Loma ) salivating ngacai / ngacay /NacaI/ Table 6: Written form variations in several local languages, confirmed by native speakers. multiple languages, therefore mixing them together. For example, people in Jember (a regency district in East Java) combine Javanese and Madurese in their daily conversation (Haryono, 2012). Indonesian code-mixing not only occurs at the word level but also at the morpheme level (Winata, 2021). For example, quotenya (his/her quote, see Table 5) combines the English word quote and the Indonesian suffix -nya , which denotes possession; similarly, ngetag combines the Betawinese prefix ngeand the English word tag . More examples can be found in Table 5. 3.3 Orthography Variation Many Indonesian local languages are mainly used in spoken settings and have no established standard orthography system. Some local languages do originally have their own archaic writing systems that derive from the Jawi alphabet or Kawi script, and even though standard transliteration into the Roman alphabet exists for some (e.g., Javanese and Sundanese), they are not widely known and practiced (Soeparno, 2015). Hence, some words have multiple romanized orthographies that are mutually intelligible, as they are pronounced the same. Some examples can be seen in Table 6. Such a variety of written forms is common in local languages in Indonesia. This variation leads to a significantly larger vocabulary size, especially for NLP systems that use word-based representations, and presents a challenge to constrain the representations for different spellings of the same word to be similar. Language evolves together with the speakers. A more widely used language may have a larger digital presence, which fosters a more written form of communication, while languages that are used only 7232 within small communities may emphasize the spoken form. Some languages are also declining, and speakers may prefer to use Indonesian rather than their local language. In contrast, there are isolated residents that use the local language daily and are less proficient in Indonesian (Nurjanah et al., 2018, Jahang and Meirina, 2021). These variations give rise to different requirements, and there is no single solution for all. Technology and education are not well-distributed within the nation. Internet penetration in Indonesia is 73.7% in 2020 but is mainly concentrated on Java. Among the non-Internet users, 39% explain that they do not understand the technology, while 15% state that they do not have a device to access the Internet. 6 In some areas where the Internet is not seen as a basic need, imposing NLP technology on them may not necessarily be relevant. At the same time, general NLP development within the nation faces difficulties due to the lack of funding, especially in universities outside of Java. GPU servers are still scarce, even in top universities in the country. 7 The dynamics of population movement in Indonesia also need to be taken into consideration. For example, urban communities transmigrate to remote areas for social purposes, such as teaching or becoming doctors for underdeveloped villages. Each of these situations might call for various new NLP technologies to be developed to facilitate better communication. Based on the challenges for Indonesian NLP highlighted in the previous section, we formulate proposals for improving the state of Indonesian NLP research, as well as of other underrepresented languages. Our proposals cover several aspects including metadata documentation; potential research directions; and engagement with communities. In line with studies promoting proper data documentation for NLP research (Bender and Friedman, 2018, Rogers et al., 2021, Alyafeai et al., 2021, McMillan-Major et al., 2022), we recommend the following considerations. Regional Dialect Metadata We have shown that a local language can have large variation depending on the region and the dialect. Therefore, we suggest adding regional dialect metadata to NLP datasets and models, not only for Indonesian but also for other languages. This is particularly important for languages with large dialectical differences. It also helps to clearly communicate NLP capabilities to stakeholders and end-users as it will help set an expectation of what dialects the systems can handle. Additionally, regional metadata can indirectly inform topics present in the data, especially for crawled data sources. Style and Register Metadata Similarly, we also suggest adding style and register metadata. This metadata can capture the politeness level of the text, not only for Indonesian but also in other languages. In addition, this metadata can be used to document the formality level of the text, so it may be useful for research on modeling style or style transfer. Among the most spoken local languages, a lot of research has been done on mainstream NLP tasks such as hate-speech detection, sentiment analysis, entity recognition, and machine translation. Some research has even been deployed in production by industry. Many of the languages, however, are not widely spoken and under-explored. Focusing on these languages, we suggest future research direction as follows. Data-Efficient NLP Pretrained language models, which have taken the NLP world by storm, require an abundance of monolingual data. However, data collection has been a long-standing problem for low-resource languages. Therefore, we recommend more exploration into designing data-efficient approaches such as adaptation methods (Artetxe et al., 2020, Aji et al., 2020, Guru-rangan et al., 2020, Koto et al., 2021, Kurniawan et al., 2021), few-shot learning (Winata et al., 2021, Madotto et al., 2021, Le Scao and Rush, 2021), and learning from related languages (Khanuja et al., 2021, Khemchandani et al., 2021). The goal of these methods is effective resource utilization, that is, to minimize the financial costs for computation and data collection as advocated by Schwartz et al. (2020), Cahyawijaya (2021), and Nityasya et al. (2021). Data Collection Data collection efforts need to be commenced as soon as possible, despite all the challenges (3.1). Here, we suggest collecting parallel data between Indonesian and each of the local languages for several reasons. First, a lot of Indonesians are bilingual (Koto and Koto, 2020), that is, they speak both Indonesian and their local language, which facilitates data collection. Moreover, the fact that the local languages have some vocabulary overlap with Indonesian (see Table 7 in the Appendix) might help facilitate building translation systems with relatively little parallel data (Nguyen and Chiang, 2017). Finally, having such parallel data, we can build translation systems for synthetic data generation. In line with this approach, the effectiveness of models trained on synthetic translated data can be explored. Compute-Efficient NLP The costly GPU requirement for current NLP models hinders adoption by local research institutions and industries. Instead of focusing on building yet another massive model, we suggest focusing on developing lightweight and fast neural architectures, for example through distillation (Kim et al., 2019; Sanh et al., 2019; Jiao et al., 2020), model factorization (Winata et al., 2019a) or model pruning (Voita et al., 2019). We also recommend research on more efficient training mechanisms (Aji and Heafield, 2017; Diskin et al., 2021). In addition, non-neural methods are still quite popular in Indonesia. Therefore, further research on the trade-off between the efficiency and quality of the models is also an interesting research direction. Robustness to Code-mixing and Non-Standard Orthography Languages in Indonesia are prone to variations due to code-mixing and non-standard orthography, which occurs on the morpheme or even grapheme level. Models that are applied to Indonesian code-mixed data need to be able to learn morphologically faithful representations. Therefore, we recommend more explorations on methods derived from subword tokenization (Gage, 1994; Kudo, 2018) and token-free models (Gillick et al., 2016; Tay et al., 2021; Xue et al., 2021a) to deal with this problem. This problem is also explored by Tan and Joty (2021) in an adversarial setting. NLP Beyond Text For many Indonesian local languages that are rarely if ever written, speech is a more natural communication format. We thus recommend more attention on less text-focused research, such as spoken language understanding (Chung et al., 2021; Serdyuk et al., 2018), speech recognition (Besacier et al., 2014; Winata et al., 2020a,b), and multimodality (Dai et al., 2020, 2021) in order to improve NLP for such languages. 8 https://inacl.id/inacl/ 9 http://polyglotindonesia.org 10 https://merajutindonesia.id/ 11 https://www.mlindonesia.org/ 12 https://bigscience.huggingface.co/ 13 https://blog.iclr.cc/2021/08/10/ broadening-our-call-for-participation-to-iclr-2022/ 7234 References Husen Abas. As discussed in 3.4, it is difficult to generalize a solution across local languages. We thus encourage the NLP community, such as the Indonesian Association of Computational Linguistics (INACL) 8 to work more closely with native speakers and local communities. Local communities who work on linguistics such as Polyglot Indonesia , 9 Merajut Indonesia , 10 and Masyarakat Linguistik Indonesia 11 would be relevant collaborators to provide solutions and resources that support use cases benefiting the native speakers and communities of underrepresented languages. We advise the involvement of linguists, for example, to aid the language documentation process (Anastasopoulos et al., 2020). We also support open-science movements such as BigScience 12 or ICLR CoSubmitting Summer 13 to help start collaborations and reduce the barrier to entry to NLP research. In this paper, we highlight challenges in Indonesian NLP. Indonesia is one of the most populous countries and the second-most linguistically diverse country of the world, with over 700 local languages, yet Indonesian NLP is underrepresented and under-explored. Based on the observed challenges, we also present recommendations to improve the situation, not only for Indonesian but also for other underrepresented languages. We are grateful to Alexander Gutkin and Ling-Yen Liao for valuable feedback on an earlier draft of this paper. We also thank the anonymous reviewers for their helpful feedback. Lastly, we acknowledge the support of Kata.ai in this work.
abstain method result objective abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method objective objective other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other other abstain abstain other abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method method abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain abstain method abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain other abstain method abstain abstain method result abstain result other other other