ayushexel/reranker-ModernBERT-base-gooaq-1-epoch-1995000

CrossEncoder based on answerdotai/ModernBERT-base

This is a Cross Encoder model finetuned from answerdotai/ModernBERT-base using the sentence-transformers library. It computes scores for pairs of texts, which can be used for text reranking and semantic search.

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: answerdotai/ModernBERT-base
• Maximum Sequence Length: 8192 tokens
• Number of Output Labels: 1 label
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import CrossEncoder

# Download from the 🤗 Hub
model = CrossEncoder("ayushexel/reranker-ModernBERT-base-gooaq-1-epoch-1995000")
# Get scores for pairs of texts
pairs = [
    ['is the beryl m762 in pubg mobile?', 'Beryl M762 is a versatile Assault Rifle in PUBG Mobile that has more attachment slots than AKM and uses 7.62 ammunition. The high damage of Beryl M762 makes it a viable option for the players.'],
    ['is the beryl m762 in pubg mobile?', 'The main difference that most people will notice while playing PUBG Mobile Lite after playing on PUBG Mobile is the availability of maps. PUBG Mobile has four maps Erangel, Miramar, Sanhok and Vikendi. PUBG Mobile Lite only has two maps; Erangel in the Classic Mode and War in the Arcade Mode.'],
    ['is the beryl m762 in pubg mobile?', 'PUBG Mobile Lite is the toned-down version of PUBG Mobile, which was developed specifically for players with low-end devices. The game is available for only Android devices at the moment, and there is no way by which you can download it on an iOS device.'],
    ['is the beryl m762 in pubg mobile?', 'Download and play PUBG Mobile on PC with NoxPlayer! PUBG Mobile is a battle royale FPS game developed by Tencent. It is similar to Garena Free Fire and Call of duty Mobile. NoxPlayer is the best emulator to play PUBG Mobile on PC.'],
    ['is the beryl m762 in pubg mobile?', "Can you play PUBG Mobile with a controller? ... For PUBG Mobile, there is no official controller support for the game outside of movement, meaning you can connect a Bluetooth-enabled controller to your mobile device and move around, but the buttons won't have any actions mapped to them."],
]
scores = model.predict(pairs)
print(scores.shape)
# (5,)

# Or rank different texts based on similarity to a single text
ranks = model.rank(
    'is the beryl m762 in pubg mobile?',
    [
        'Beryl M762 is a versatile Assault Rifle in PUBG Mobile that has more attachment slots than AKM and uses 7.62 ammunition. The high damage of Beryl M762 makes it a viable option for the players.',
        'The main difference that most people will notice while playing PUBG Mobile Lite after playing on PUBG Mobile is the availability of maps. PUBG Mobile has four maps Erangel, Miramar, Sanhok and Vikendi. PUBG Mobile Lite only has two maps; Erangel in the Classic Mode and War in the Arcade Mode.',
        'PUBG Mobile Lite is the toned-down version of PUBG Mobile, which was developed specifically for players with low-end devices. The game is available for only Android devices at the moment, and there is no way by which you can download it on an iOS device.',
        'Download and play PUBG Mobile on PC with NoxPlayer! PUBG Mobile is a battle royale FPS game developed by Tencent. It is similar to Garena Free Fire and Call of duty Mobile. NoxPlayer is the best emulator to play PUBG Mobile on PC.',
        "Can you play PUBG Mobile with a controller? ... For PUBG Mobile, there is no official controller support for the game outside of movement, meaning you can connect a Bluetooth-enabled controller to your mobile device and move around, but the buttons won't have any actions mapped to them.",
    ]
)
# [{'corpus_id': ..., 'score': ...}, {'corpus_id': ..., 'score': ...}, ...]

Evaluation

Metrics

Cross Encoder Reranking

• Dataset: gooaq-dev
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:

  {
      "at_k": 10,
      "always_rerank_positives": false
  }
  

Metric	Value
map	0.4829 (+0.2133)
mrr@10	0.4823 (+0.2235)
ndcg@10	0.5236 (+0.2141)

Cross Encoder Reranking

• Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:

  {
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	NanoMSMARCO_R100	NanoNFCorpus_R100	NanoNQ_R100
map	0.4301 (-0.0595)	0.3684 (+0.1074)	0.4224 (+0.0028)
mrr@10	0.4149 (-0.0626)	0.4482 (-0.0516)	0.4220 (-0.0047)
ndcg@10	0.4928 (-0.0477)	0.3782 (+0.0531)	0.4617 (-0.0390)

Cross Encoder Nano BEIR

• Dataset: NanoBEIR_R100_mean
• Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:

  {
      "dataset_names": [
          "msmarco",
          "nfcorpus",
          "nq"
      ],
      "rerank_k": 100,
      "at_k": 10,
      "always_rerank_positives": true
  }
  

Metric	Value
map	0.4070 (+0.0169)
mrr@10	0.4284 (-0.0396)
ndcg@10	0.4442 (-0.0112)

Training Details

Training Dataset

Unnamed Dataset

• Size: 11,456,701 training samples
• Columns: question, answer, and label
• Approximate statistics based on the first 1000 samples:

  	question	answer	label
  type	string	string	int
  details	min: 20 characters mean: 44.38 characters max: 82 characters	min: 57 characters mean: 253.74 characters max: 358 characters	0: ~82.70% 1: ~17.30%

• Samples:

  question	answer	label
  is the beryl m762 in pubg mobile?	Beryl M762 is a versatile Assault Rifle in PUBG Mobile that has more attachment slots than AKM and uses 7.62 ammunition. The high damage of Beryl M762 makes it a viable option for the players.	1
  is the beryl m762 in pubg mobile?	The main difference that most people will notice while playing PUBG Mobile Lite after playing on PUBG Mobile is the availability of maps. PUBG Mobile has four maps Erangel, Miramar, Sanhok and Vikendi. PUBG Mobile Lite only has two maps; Erangel in the Classic Mode and War in the Arcade Mode.	0
  is the beryl m762 in pubg mobile?	PUBG Mobile Lite is the toned-down version of PUBG Mobile, which was developed specifically for players with low-end devices. The game is available for only Android devices at the moment, and there is no way by which you can download it on an iOS device.	0

• Loss: BinaryCrossEntropyLoss with these parameters:

  {
      "activation_fn": "torch.nn.modules.linear.Identity",
      "pos_weight": 5
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• dataloader_num_workers: 12
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 12
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 12
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• tp_size: 0
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	gooaq-dev_ndcg@10	NanoMSMARCO_R100_ndcg@10	NanoNFCorpus_R100_ndcg@10	NanoNQ_R100_ndcg@10	NanoBEIR_R100_mean_ndcg@10
-1	-1	-	0.1056 (-0.2039)	0.0327 (-0.5077)	0.2403 (-0.0847)	0.0253 (-0.4753)	0.0995 (-0.3559)
0.0000	1	1.1878	-	-	-	-	-
0.0045	200	1.2005	-	-	-	-	-
0.0089	400	1.1777	-	-	-	-	-
0.0134	600	1.1557	-	-	-	-	-
0.0179	800	1.0045	-	-	-	-	-
0.0223	1000	0.7861	-	-	-	-	-
0.0268	1200	0.7065	-	-	-	-	-
0.0313	1400	0.6585	-	-	-	-	-
0.0358	1600	0.6381	-	-	-	-	-
0.0402	1800	0.6047	-	-	-	-	-
0.0447	2000	0.594	-	-	-	-	-
0.0492	2200	0.5911	-	-	-	-	-
0.0536	2400	0.5652	-	-	-	-	-
0.0581	2600	0.5541	-	-	-	-	-
0.0626	2800	0.5445	-	-	-	-	-
0.0670	3000	0.5234	-	-	-	-	-
0.0715	3200	0.5215	-	-	-	-	-
0.0760	3400	0.5297	-	-	-	-	-
0.0804	3600	0.517	-	-	-	-	-
0.0849	3800	0.5108	-	-	-	-	-
0.0894	4000	0.5051	-	-	-	-	-
0.0938	4200	0.5114	-	-	-	-	-
0.0983	4400	0.5026	-	-	-	-	-
0.1028	4600	0.4826	-	-	-	-	-
0.1073	4800	0.489	-	-	-	-	-
0.1117	5000	0.4845	-	-	-	-	-
0.1162	5200	0.4827	-	-	-	-	-
0.1207	5400	0.4715	-	-	-	-	-
0.1251	5600	0.4714	-	-	-	-	-
0.1296	5800	0.4606	-	-	-	-	-
0.1341	6000	0.4629	-	-	-	-	-
0.1385	6200	0.4488	-	-	-	-	-
0.1430	6400	0.4605	-	-	-	-	-
0.1475	6600	0.4622	-	-	-	-	-
0.1519	6800	0.4555	-	-	-	-	-
0.1564	7000	0.4512	-	-	-	-	-
0.1609	7200	0.4403	-	-	-	-	-
0.1654	7400	0.44	-	-	-	-	-
0.1698	7600	0.4444	-	-	-	-	-
0.1743	7800	0.441	-	-	-	-	-
0.1788	8000	0.4364	-	-	-	-	-
0.1832	8200	0.4419	-	-	-	-	-
0.1877	8400	0.4283	-	-	-	-	-
0.1922	8600	0.4296	-	-	-	-	-
0.1966	8800	0.419	-	-	-	-	-
0.2011	9000	0.4385	-	-	-	-	-
0.2056	9200	0.4282	-	-	-	-	-
0.2100	9400	0.4171	-	-	-	-	-
0.2145	9600	0.4249	-	-	-	-	-
0.2190	9800	0.4191	-	-	-	-	-
0.2234	10000	0.4227	-	-	-	-	-
0.2279	10200	0.4179	-	-	-	-	-
0.2324	10400	0.4203	-	-	-	-	-
0.2369	10600	0.4125	-	-	-	-	-
0.2413	10800	0.4065	-	-	-	-	-
0.2458	11000	0.4068	-	-	-	-	-
0.2503	11200	0.4122	-	-	-	-	-
0.2547	11400	0.4217	-	-	-	-	-
0.2592	11600	0.4054	-	-	-	-	-
0.2637	11800	0.4004	-	-	-	-	-
0.2681	12000	0.4143	-	-	-	-	-
0.2726	12200	0.4012	-	-	-	-	-
0.2771	12400	0.4023	-	-	-	-	-
0.2815	12600	0.3975	-	-	-	-	-
0.2860	12800	0.3983	-	-	-	-	-
0.2905	13000	0.3958	-	-	-	-	-
0.2950	13200	0.4009	-	-	-	-	-
0.2994	13400	0.4048	-	-	-	-	-
0.3039	13600	0.4017	-	-	-	-	-
0.3084	13800	0.4009	-	-	-	-	-
0.3128	14000	0.3865	-	-	-	-	-
0.3173	14200	0.3883	-	-	-	-	-
0.3218	14400	0.3864	-	-	-	-	-
0.3262	14600	0.3873	-	-	-	-	-
0.3307	14800	0.3906	-	-	-	-	-
0.3352	15000	0.3909	-	-	-	-	-
0.3396	15200	0.3881	-	-	-	-	-
0.3441	15400	0.3814	-	-	-	-	-
0.3486	15600	0.3813	-	-	-	-	-
0.3530	15800	0.3785	-	-	-	-	-
0.3575	16000	0.3852	-	-	-	-	-
0.3620	16200	0.381	-	-	-	-	-
0.3665	16400	0.3865	-	-	-	-	-
0.3709	16600	0.377	-	-	-	-	-
0.3754	16800	0.3762	-	-	-	-	-
0.3799	17000	0.379	-	-	-	-	-
0.3843	17200	0.3738	-	-	-	-	-
0.3888	17400	0.38	-	-	-	-	-
0.3933	17600	0.3786	-	-	-	-	-
0.3977	17800	0.3825	-	-	-	-	-
0.4022	18000	0.3691	-	-	-	-	-
0.4067	18200	0.3732	-	-	-	-	-
0.4111	18400	0.3777	-	-	-	-	-
0.4156	18600	0.378	-	-	-	-	-
0.4201	18800	0.3627	-	-	-	-	-
0.4246	19000	0.3698	-	-	-	-	-
0.4290	19200	0.3746	-	-	-	-	-
0.4335	19400	0.3868	-	-	-	-	-
0.4380	19600	0.3659	-	-	-	-	-
0.4424	19800	0.3713	-	-	-	-	-
0.4469	20000	0.3685	-	-	-	-	-
0.4514	20200	0.3737	-	-	-	-	-
0.4558	20400	0.3653	-	-	-	-	-
0.4603	20600	0.3648	-	-	-	-	-
0.4648	20800	0.3684	-	-	-	-	-
0.4692	21000	0.3638	-	-	-	-	-
0.4737	21200	0.3628	-	-	-	-	-
0.4782	21400	0.3662	-	-	-	-	-
0.4826	21600	0.3662	-	-	-	-	-
0.4871	21800	0.3696	-	-	-	-	-
0.4916	22000	0.3664	-	-	-	-	-
0.4961	22200	0.3583	-	-	-	-	-
0.5005	22400	0.3666	-	-	-	-	-
0.5050	22600	0.3637	-	-	-	-	-
0.5095	22800	0.3679	-	-	-	-	-
0.5139	23000	0.3609	-	-	-	-	-
0.5184	23200	0.3566	-	-	-	-	-
0.5229	23400	0.3573	-	-	-	-	-
0.5273	23600	0.3576	-	-	-	-	-
0.5318	23800	0.3566	-	-	-	-	-
0.5363	24000	0.3541	-	-	-	-	-
0.5407	24200	0.3498	-	-	-	-	-
0.5452	24400	0.3462	-	-	-	-	-
0.5497	24600	0.3484	-	-	-	-	-
0.5542	24800	0.3461	-	-	-	-	-
0.5586	25000	0.3517	-	-	-	-	-
0.5631	25200	0.3494	-	-	-	-	-
0.5676	25400	0.3487	-	-	-	-	-
0.5720	25600	0.3447	-	-	-	-	-
0.5765	25800	0.3531	-	-	-	-	-
0.5810	26000	0.3515	-	-	-	-	-
0.5854	26200	0.3498	-	-	-	-	-
0.5899	26400	0.3491	-	-	-	-	-
0.5944	26600	0.3486	-	-	-	-	-
0.5988	26800	0.3498	-	-	-	-	-
0.6033	27000	0.3461	-	-	-	-	-
0.6078	27200	0.3482	-	-	-	-	-
0.6122	27400	0.3492	-	-	-	-	-
0.6167	27600	0.3455	-	-	-	-	-
0.6212	27800	0.3509	-	-	-	-	-
0.6257	28000	0.3477	-	-	-	-	-
0.6301	28200	0.3485	-	-	-	-	-
0.6346	28400	0.3474	-	-	-	-	-
0.6391	28600	0.3407	-	-	-	-	-
0.6435	28800	0.3398	-	-	-	-	-
0.6480	29000	0.3444	-	-	-	-	-
0.6525	29200	0.3357	-	-	-	-	-
0.6569	29400	0.3481	-	-	-	-	-
0.6614	29600	0.3375	-	-	-	-	-
0.6659	29800	0.341	-	-	-	-	-
0.6703	30000	0.3388	-	-	-	-	-
0.6748	30200	0.329	-	-	-	-	-
0.6793	30400	0.3394	-	-	-	-	-
0.6838	30600	0.3535	-	-	-	-	-
0.6882	30800	0.3436	-	-	-	-	-
0.6927	31000	0.3455	-	-	-	-	-
0.6972	31200	0.3319	-	-	-	-	-
0.7016	31400	0.3376	-	-	-	-	-
0.7061	31600	0.337	-	-	-	-	-
0.7106	31800	0.3387	-	-	-	-	-
0.7150	32000	0.3398	-	-	-	-	-
0.7195	32200	0.3359	-	-	-	-	-
0.7240	32400	0.3327	-	-	-	-	-
0.7284	32600	0.3343	-	-	-	-	-
0.7329	32800	0.3285	-	-	-	-	-
0.7374	33000	0.3332	-	-	-	-	-
0.7418	33200	0.3291	-	-	-	-	-
0.7463	33400	0.3445	-	-	-	-	-
0.7508	33600	0.3372	-	-	-	-	-
0.7553	33800	0.3258	-	-	-	-	-
0.7597	34000	0.3352	-	-	-	-	-
0.7642	34200	0.3344	-	-	-	-	-
0.7687	34400	0.329	-	-	-	-	-
0.7731	34600	0.3301	-	-	-	-	-
0.7776	34800	0.3312	-	-	-	-	-
0.7821	35000	0.3242	-	-	-	-	-
0.7865	35200	0.3349	-	-	-	-	-
0.7910	35400	0.3288	-	-	-	-	-
0.7955	35600	0.3289	-	-	-	-	-
0.7999	35800	0.3209	-	-	-	-	-
0.8044	36000	0.3279	-	-	-	-	-
0.8089	36200	0.3274	-	-	-	-	-
0.8134	36400	0.3355	-	-	-	-	-
0.8178	36600	0.3265	-	-	-	-	-
0.8223	36800	0.3263	-	-	-	-	-
0.8268	37000	0.3301	-	-	-	-	-
0.8312	37200	0.3209	-	-	-	-	-
0.8357	37400	0.3172	-	-	-	-	-
0.8402	37600	0.332	-	-	-	-	-
0.8446	37800	0.3345	-	-	-	-	-
0.8491	38000	0.3311	-	-	-	-	-
0.8536	38200	0.3208	-	-	-	-	-
0.8580	38400	0.3301	-	-	-	-	-
0.8625	38600	0.3279	-	-	-	-	-
0.8670	38800	0.3251	-	-	-	-	-
0.8714	39000	0.3264	-	-	-	-	-
0.8759	39200	0.3247	-	-	-	-	-
0.8804	39400	0.3267	-	-	-	-	-
0.8849	39600	0.3311	-	-	-	-	-
0.8893	39800	0.3218	-	-	-	-	-
0.8938	40000	0.3249	-	-	-	-	-
0.8983	40200	0.3314	-	-	-	-	-
0.9027	40400	0.3189	-	-	-	-	-
0.9072	40600	0.3187	-	-	-	-	-
0.9117	40800	0.3154	-	-	-	-	-
0.9161	41000	0.3206	-	-	-	-	-
0.9206	41200	0.3251	-	-	-	-	-
0.9251	41400	0.3236	-	-	-	-	-
0.9295	41600	0.3292	-	-	-	-	-
0.9340	41800	0.3309	-	-	-	-	-
0.9385	42000	0.3204	-	-	-	-	-
0.9430	42200	0.3223	-	-	-	-	-
0.9474	42400	0.3155	-	-	-	-	-
0.9519	42600	0.322	-	-	-	-	-
0.9564	42800	0.3204	-	-	-	-	-
0.9608	43000	0.3249	-	-	-	-	-
0.9653	43200	0.3244	-	-	-	-	-
0.9698	43400	0.3208	-	-	-	-	-
0.9742	43600	0.3295	-	-	-	-	-
0.9787	43800	0.3283	-	-	-	-	-
0.9832	44000	0.3188	-	-	-	-	-
0.9876	44200	0.321	-	-	-	-	-
0.9921	44400	0.3178	-	-	-	-	-
0.9966	44600	0.326	-	-	-	-	-
-1	-1	-	0.5236 (+0.2141)	0.4928 (-0.0477)	0.3782 (+0.0531)	0.4617 (-0.0390)	0.4442 (-0.0112)

Framework Versions

• Python: 3.11.0
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}