XLM-R Adapter-Based Multitask Model (customized-mwt-ner
)
This repository contains a fine-tuned XLM-RoBERTa model enhanced with task-specific adapter layers for multilingual NLP tasks including:
- Part-of-Speech tagging (
UPOS
,XPOS
) - Morphological features tagging (
FEATS
) - Dependency parsing (
DEPREL
,HEAD
) - Named Entity Recognition (NER)
Trained using adapter-based finetuning and multi-task supervision on Universal Dependencies and custom NER data.
Files
customized-mwt-ner.tagger.mdl
→ PyTorch state dict containing task adapter weights and classification heads.customized-mwt-ner.vocabs.json
→ Token and label vocabulary mappings (used for preprocessing and decoding).
Model Architecture
- Base:
xlm-roberta-base
(frozen) - Adapter configuration:
Pfeiffer
- Injected adapter layer per Transformer block
- Task-specific heads for:
- POS (
upos
,xpos
) - Morphological features (
feats
) - Dependency parsing (
unlabeled
,deprel
) - Optional NER layer (if enabled)
- POS (
Usage (PyTorch)
You’ll need to:
- Load
xlm-roberta-base
from Hugging Face - Load adapter weights from
customized-mwt-ner.tagger.mdl
- Use the vocab file for decoding predictions
NOTE: This model is not directly plug-and-play with
transformers
unless you recreate the original architecture and adapter insertion logic.
Refer to the original paper for complete documentation.
Variants
This model comes in four variants:
Context | Label Granularity | Folder / Repo Name |
---|---|---|
With Context | Coarse | with-context-coarse |
With Context | Fine-grain | with-context-finegrain |
Without Context | Coarse | without-context-coarse |
Without Context | Fine-grain | without-context-finegrain |
License
APACHE 2.0 License. Please cite appropriately if used in research.
Citation
@misc{sandhan2023depnecti,
title={DepNeCTI: Dependency-based Nested Compound Type Identification for Sanskrit},
author={Jivnesh Sandhan and Yaswanth Narsupalli and Sreevatsa Muppirala and Sriram Krishnan and Pavankumar Satuluri and Amba Kulkarni and Pawan Goyal},
year={2023},
eprint={2310.09501},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Link to original paper => HERE
Link to DepNeCT Github Repo => HERE