roberta-base-atomic.train.no.negation.true.irrelevant1e-06-64

This model is a fine-tuned version of roberta-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4115

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 256
• eval_batch_size: 1024
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss
0.5529	1.0	795	0.4975
0.4983	2.0	1590	0.4678
0.4798	3.0	2385	0.4578
0.4627	4.0	3180	0.4466
0.4588	5.0	3975	0.4402
0.4464	6.0	4770	0.4374
0.4416	7.0	5565	0.4325
0.4364	8.0	6360	0.4280
0.4341	9.0	7155	0.4276
0.4251	10.0	7950	0.4265
0.4204	11.0	8745	0.4225
0.4179	12.0	9540	0.4236
0.4158	13.0	10335	0.4199
0.4135	14.0	11130	0.4192
0.4097	15.0	11925	0.4173
0.4058	16.0	12720	0.4181
0.4064	17.0	13515	0.4158
0.4021	18.0	14310	0.4148
0.4006	19.0	15105	0.4143
0.3978	20.0	15900	0.4118
0.3971	21.0	16695	0.4149
0.3965	22.0	17490	0.4125
0.3939	23.0	18285	0.4113
0.393	24.0	19080	0.4141
0.3904	25.0	19875	0.4126
0.3963	26.0	20670	0.4113
0.3944	27.0	21465	0.4106
0.3918	28.0	22260	0.4121
0.3896	29.0	23055	0.4115
0.3905	30.0	23850	0.4115

Framework versions

• Transformers 4.51.2
• Pytorch 2.6.0+cu124
• Datasets 3.5.0
• Tokenizers 0.21.1