--- language: - en license: cc-by-sa-4.0 library_name: span-marker tags: - span-marker - token-classification - ner - named-entity-recognition - generated_from_span_marker_trainer datasets: - tomaarsen/ner-orgs metrics: - precision - recall - f1 widget: - text: In 2005, Shankel signed with Warner Chappell Music and while pursuing his own projects created another joint venture, Shankel Songs and signed Ben Glover, "Billboard "'s Christian writer of the Year, 2010, Joy Williams of The Civil Wars, and, whom he also produced. - text: In 2002, Rodríguez moved to Mississippi and to the NASA Stennis Space Center as the Director of Center Operations and as a member of the Senior Executive Service where he managed facility construction, security and other programs for 4,500 Stennis personnel. - text: American Motors included Chinese officials as part of the negotiations establishing Beijing Jeep (now Beijing Benz). - text: La Señora () is a popular Spanish television period drama series set in the 1920s, produced by Diagonal TV for Televisión Española that was broadcast on La 1 of Televisión Española from 2008 to 2010. - text: 'Not only did the Hungarian Ministry of Foreign Affairs approve Radio Free Europe''s new location, but the Ministry of Telecommunications did something even more amazing: "They found us four phone lines in central Budapest," says Geza Szocs, a Radio Free Europe correspondent who helped organize the Budapest location.' pipeline_tag: token-classification co2_eq_emissions: emissions: 67.93561835707102 source: codecarbon training_type: fine-tuning on_cloud: false cpu_model: 13th Gen Intel(R) Core(TM) i7-13700K ram_total_size: 31.777088165283203 hours_used: 0.52 hardware_used: 1 x NVIDIA GeForce RTX 3090 base_model: prajjwal1/bert-small model-index: - name: SpanMarker with prajjwal1/bert-small on FewNERD, CoNLL2003, and OntoNotes v5 results: - task: type: token-classification name: Named Entity Recognition dataset: name: FewNERD, CoNLL2003, and OntoNotes v5 type: tomaarsen/ner-orgs split: test metrics: - type: f1 value: 0.7547025470254703 name: F1 - type: precision value: 0.7617641715116279 name: Precision - type: recall value: 0.7477706438380596 name: Recall --- # SpanMarker with prajjwal1/bert-small on FewNERD, CoNLL2003, and OntoNotes v5 This is a [SpanMarker](https://github.com/tomaarsen/SpanMarkerNER) model trained on the [FewNERD, CoNLL2003, and OntoNotes v5](https://huggingface.co/datasets/tomaarsen/ner-orgs) dataset that can be used for Named Entity Recognition. This SpanMarker model uses [prajjwal1/bert-small](https://huggingface.co/prajjwal1/bert-small) as the underlying encoder. ## Model Details ### Model Description - **Model Type:** SpanMarker - **Encoder:** [prajjwal1/bert-small](https://huggingface.co/prajjwal1/bert-small) - **Maximum Sequence Length:** 256 tokens - **Maximum Entity Length:** 8 words - **Training Dataset:** [FewNERD, CoNLL2003, and OntoNotes v5](https://huggingface.co/datasets/tomaarsen/ner-orgs) - **Language:** en - **License:** cc-by-sa-4.0 ### Model Sources - **Repository:** [SpanMarker on GitHub](https://github.com/tomaarsen/SpanMarkerNER) - **Thesis:** [SpanMarker For Named Entity Recognition](https://raw.githubusercontent.com/tomaarsen/SpanMarkerNER/main/thesis.pdf) ### Model Labels | Label | Examples | |:------|:---------------------------------------------| | ORG | "Texas Chicken", "Church 's Chicken", "IAEA" | ## Evaluation ### Metrics | Label | Precision | Recall | F1 | |:--------|:----------|:-------|:-------| | **all** | 0.7618 | 0.7478 | 0.7547 | | ORG | 0.7618 | 0.7478 | 0.7547 | ## Uses ### Direct Use for Inference ```python from span_marker import SpanMarkerModel # Download from the 🤗 Hub model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-small-orgs") # Run inference entities = model.predict("American Motors included Chinese officials as part of the negotiations establishing Beijing Jeep (now Beijing Benz).") ``` ### Downstream Use You can finetune this model on your own dataset.
Click to expand ```python from span_marker import SpanMarkerModel, Trainer # Download from the 🤗 Hub model = SpanMarkerModel.from_pretrained("tomaarsen/span-marker-bert-small-orgs") # Specify a Dataset with "tokens" and "ner_tag" columns dataset = load_dataset("conll2003") # For example CoNLL2003 # Initialize a Trainer using the pretrained model & dataset trainer = Trainer( model=model, train_dataset=dataset["train"], eval_dataset=dataset["validation"], ) trainer.train() trainer.save_model("tomaarsen/span-marker-bert-small-orgs-finetuned") ```
## Training Details ### Training Set Metrics | Training set | Min | Median | Max | |:----------------------|:----|:--------|:----| | Sentence length | 1 | 23.5706 | 263 | | Entities per sentence | 0 | 0.7865 | 39 | ### Training Hyperparameters - learning_rate: 0.0001 - train_batch_size: 128 - eval_batch_size: 128 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_ratio: 0.1 - num_epochs: 3 ### Training Results | Epoch | Step | Validation Loss | Validation Precision | Validation Recall | Validation F1 | Validation Accuracy | |:------:|:----:|:---------------:|:--------------------:|:-----------------:|:-------------:|:-------------------:| | 0.5720 | 600 | 0.0076 | 0.7642 | 0.6630 | 0.7100 | 0.9656 | | 1.1439 | 1200 | 0.0070 | 0.7705 | 0.7139 | 0.7411 | 0.9699 | | 1.7159 | 1800 | 0.0067 | 0.7837 | 0.7231 | 0.7522 | 0.9709 | | 2.2879 | 2400 | 0.0070 | 0.7768 | 0.7517 | 0.7640 | 0.9725 | | 2.8599 | 3000 | 0.0068 | 0.7877 | 0.7374 | 0.7617 | 0.9718 | ### Environmental Impact Carbon emissions were measured using [CodeCarbon](https://github.com/mlco2/codecarbon). - **Carbon Emitted**: 0.068 kg of CO2 - **Hours Used**: 0.52 hours ### Training Hardware - **On Cloud**: No - **GPU Model**: 1 x NVIDIA GeForce RTX 3090 - **CPU Model**: 13th Gen Intel(R) Core(TM) i7-13700K - **RAM Size**: 31.78 GB ### Framework Versions - Python: 3.9.16 - SpanMarker: 1.5.1.dev - Transformers: 4.30.0 - PyTorch: 2.0.1+cu118 - Datasets: 2.14.0 - Tokenizers: 0.13.3 ## Citation ### BibTeX ``` @software{Aarsen_SpanMarker, author = {Aarsen, Tom}, license = {Apache-2.0}, title = {{SpanMarker for Named Entity Recognition}}, url = {https://github.com/tomaarsen/SpanMarkerNER} } ```