readme: ---
  language:
  - as
  - bn
  - gu
  - hi
  - mr
  - ne
  - or
  - pa
  - si
  - sa
  - bpy
  - mai
  - bh
  - gom
  license: apache-2.0
  datasets:
  - oscar
  tags:
  - multilingual
  - albert
  - xlmindic
  - nlp
  - indoaryan
  - indicnlp
  - iso15919
  - transliteration
  - text-classification
  widget:
  - text : 'cīnēra madhyāñcalē āraō ēkaṭi śaharēra bāsindārā ābāra gharabandī haẏē paṛēchēna. āja maṅgalabāra natuna karē lakaḍāuna–saṁkrānta bidhiniṣēdha jāri haōẏāra para gharē āṭakā paṛēchēna tām̐rā. karōnāra ati saṁkrāmaka natuna dharana amikranēra bistāra ṭhēkātē ēmana padakṣēpa niẏēchē kartr̥pakṣa. khabara bārtā saṁsthā ēēphapira.'
  
  co2_eq_emissions:
        emissions: "0.21 in grams of CO2"
        source: "calculated using this webstie https://mlco2.github.io/impact/#compute"
        training_type: "fine-tuning"
        geographical_location: "NA"
        hardware_used: "P100 for about 1.5 hours"
  ---
  
  # XLMIndic Base Uniscript
  
  This model is finetuned from [this model](https://huggingface.co/ibraheemmoosa/xlmindic-base-uniscript) on Soham Bangla News Classification task which is part of the IndicGLUE benchmark. **Before pretraining this model we transliterate the text to [ISO-15919](https://en.wikipedia.org/wiki/ISO_15919) format using the [Aksharamukha](https://pypi.org/project/aksharamukha/)
  library.** A demo of Aksharamukha library is hosted [here](https://aksharamukha.appspot.com/converter)
  where you can transliterate your text and use it on our model on the inference widget.
  
  ## Model description
  
  This model has the same configuration as the [ALBERT Base v2 model](https://huggingface.co/albert-base-v2/). Specifically, this model has the following configuration:
  
  - 12 repeating layers
  - 128 embedding dimension
  - 768 hidden dimension
  - 12 attention heads
  - 11M parameters
  - 512 sequence length
  
  ## Training data
  This model was fine-tuned on Soham dataset that is part of the IndicGLUE benchmark.
  
  ## Transliteration
  
  *The unique component of this model is that it takes in ISO-15919 transliterated text.*
  
  The motivation behind this is this. When two languages share vocabularies, a machine learning model can exploit that to learn good cross-lingual representations. However if these two languages use different writing scripts it is difficult for a model to make the connection. Thus if if we can write the two languages in a single script then it is easier for the model to learn good cross-lingual representation.
  
  For many of the scripts currently in use, there are standard transliteration schemes to convert to the Latin script. In particular, for the Indic scripts the ISO-15919 transliteration scheme is designed to consistently transliterate texts written in different Indic scripts to the Latin script.
  
  An example of ISO-15919 transliteration for a piece of **Bangla** text is the following:
  
  **Original:** "রবীন্দ্রনাথ ঠাকুর এফআরএএস (৭ মে ১৮৬১ - ৭ আগস্ট ১৯৪১; ২৫ বৈশাখ ১২৬৮ - ২২ শ্রাবণ ১৩৪৮ বঙ্গাব্দ) ছিলেন অগ্রণী বাঙালি কবি, ঔপন্যাসিক, সংগীতস্রষ্টা, নাট্যকার, চিত্রকর, ছোটগল্পকার, প্রাবন্ধিক, অভিনেতা, কণ্ঠশিল্পী ও দার্শনিক।"
  
  **Transliterated:** 'rabīndranātha ṭhākura ēphaāraēēsa (7 mē 1861 - 7 āgasṭa 1941; 25 baiśākha 1268 - 22 śrābaṇa 1348 baṅgābda) chilēna agraṇī bāṅāli kabi, aupanyāsika, saṁgītasraṣṭā, nāṭyakāra, citrakara, chōṭagalpakāra, prābandhika, abhinētā, kaṇṭhaśilpī ō dārśanika.'
  
  Another example for a piece of **Hindi** text is the following:
  
  **Original:** "चूंकि मानव परिवार के सभी सदस्यों के जन्मजात गौरव और समान तथा अविच्छिन्न अधिकार की स्वीकृति ही विश्व-शान्ति, न्याय और स्वतन्त्रता की बुनियाद है"
  
  **Transliterated:** "cūṁki mānava parivāra kē sabhī sadasyōṁ kē janmajāta gaurava aura samāna tathā avicchinna adhikāra kī svīkr̥ti hī viśva-śānti, nyāya aura svatantratā kī buniyāda hai"
   
   
  ## Training procedure
  
  ### Preprocessing
  
  The texts are transliterated to ISO-15919 format using the Aksharamukha library. Then these are tokenized using SentencePiece and a vocabulary size of 50,000.
  
  ### Training
  
  The model was trained for 8 epochs with a batch size of 16 and a learning rate of *2e-5*.
  
  ## Evaluation results
  See results specific to Soham in the following table.
  
  ### IndicGLUE
  Task | mBERT | XLM-R | IndicBERT-Base | XLMIndic-Base-Uniscript (This Model) | XLMIndic-Base-Multiscript (Ablation Model)
  -----| ----- | ----- | ------ | ------- | --------
  Wikipedia Section Title Prediction | 71.90 | 65.45 | 69.40 | **81.78 ± 0.60** | 77.17 ± 0.76
  Article Genre Classification | 88.64 | 96.61 | 97.72 | **98.70 ± 0.29** | 98.30 ± 0.26
  Named Entity Recognition (F1-score) | 71.29 | 62.18 | 56.69 | **89.85 ± 1.14** |  83.19 ± 1.58
  BBC Hindi News Article Classification | 60.55 | 75.52 | 74.60 | **79.14 ± 0.60** | 77.28 ± 1.50
  Soham Bangla News Article Classification | 80.23 | 87.6 | 78.45 | **93.89 ± 0.48** | 93.22 ± 0.49
  INLTK Gujarati Headlines Genre Classification | - | - | **92.91** | 90.73 ± 0.75 | 90.41 ± 0.69
  INLTK Marathi Headlines Genre Classification | - | - | **94.30** | 92.04 ± 0.47 | 92.21 ± 0.23
  IITP Hindi Product Reviews Sentiment Classification | 74.57 | 