XLM-R Adapter-Based Multitask Model (customized-mwt-ner)

This repository contains a fine-tuned XLM-RoBERTa model enhanced with task-specific adapter layers for multilingual NLP tasks including:

  • Part-of-Speech tagging (UPOS, XPOS)
  • Morphological features tagging (FEATS)
  • Dependency parsing (DEPREL, HEAD)
  • Named Entity Recognition (NER)

Trained using adapter-based finetuning and multi-task supervision on Universal Dependencies and custom NER data.


Files

  • customized-mwt-ner.tagger.mdl
    → PyTorch state dict containing task adapter weights and classification heads.

  • customized-mwt-ner.vocabs.json
    → Token and label vocabulary mappings (used for preprocessing and decoding).


Model Architecture

  • Base: xlm-roberta-base (frozen)
  • Adapter configuration: Pfeiffer
  • Injected adapter layer per Transformer block
  • Task-specific heads for:
    • POS (upos, xpos)
    • Morphological features (feats)
    • Dependency parsing (unlabeled, deprel)
    • Optional NER layer (if enabled)

Usage (PyTorch)

You’ll need to:

  1. Load xlm-roberta-base from Hugging Face
  2. Load adapter weights from customized-mwt-ner.tagger.mdl
  3. Use the vocab file for decoding predictions

NOTE: This model is not directly plug-and-play with transformers unless you recreate the original architecture and adapter insertion logic.

Refer to the original paper for complete documentation.


Variants

This model comes in four variants:

Context Label Granularity Folder / Repo Name
With Context Coarse with-context-coarse
With Context Fine-grain with-context-finegrain
Without Context Coarse without-context-coarse
Without Context Fine-grain without-context-finegrain

License

APACHE 2.0 License. Please cite appropriately if used in research.


Citation

@misc{sandhan2023depnecti,
      title={DepNeCTI: Dependency-based Nested Compound Type Identification for Sanskrit}, 
      author={Jivnesh Sandhan and Yaswanth Narsupalli and Sreevatsa Muppirala and Sriram Krishnan and Pavankumar Satuluri and Amba Kulkarni and Pawan Goyal},
      year={2023},
      eprint={2310.09501},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Link to original paper => HERE
Link to DepNeCT Github Repo => HERE

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train sanganaka/DepNeCT-XLMR

Collection including sanganaka/DepNeCT-XLMR