camembert_ccnet_classification_tools_classifier-only_fr_V2

This model is a fine-tuned version of camembert/camembert-base-ccnet on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.8155
• Accuracy: 0.7396
• Learning Rate: 0.0000

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 24
• eval_batch_size: 192
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Rate
2.0589	1.0	15	2.0475	0.25	0.0001
2.0002	2.0	30	1.9708	0.3958	0.0001
1.9338	3.0	45	1.9353	0.3646	0.0001
1.8829	4.0	60	1.8915	0.3958	0.0001
1.8255	5.0	75	1.8285	0.4688	0.0001
1.79	6.0	90	1.7780	0.5833	9e-05
1.7391	7.0	105	1.7396	0.4688	0.0001
1.6838	8.0	120	1.6781	0.5417	0.0001
1.6385	9.0	135	1.6208	0.6042	0.0001
1.6083	10.0	150	1.5847	0.5729	0.0001
1.5543	11.0	165	1.5476	0.625	0.0001
1.5171	12.0	180	1.5068	0.6042	8e-05
1.4848	13.0	195	1.4651	0.6354	0.0001
1.4068	14.0	210	1.4436	0.6354	0.0001
1.4141	15.0	225	1.3965	0.6042	0.0001
1.3678	16.0	240	1.3565	0.6562	0.0001
1.309	17.0	255	1.3221	0.6875	0.0001
1.2867	18.0	270	1.2940	0.6667	7e-05
1.2667	19.0	285	1.2548	0.6667	0.0001
1.2271	20.0	300	1.2286	0.6875	0.0001
1.1865	21.0	315	1.1943	0.6875	0.0001
1.1691	22.0	330	1.1705	0.6979	0.0001
1.1502	23.0	345	1.1483	0.6875	0.0001
1.1195	24.0	360	1.1250	0.6875	6e-05
1.1148	25.0	375	1.1058	0.6979	0.0001
1.0893	26.0	390	1.0787	0.7083	0.0001
1.0736	27.0	405	1.0668	0.6979	0.0001
1.0303	28.0	420	1.0466	0.6979	0.0001
1.0475	29.0	435	1.0211	0.6979	0.0001
1.0122	30.0	450	0.9995	0.7292	5e-05
1.0014	31.0	465	0.9907	0.7292	0.0000
0.973	32.0	480	0.9791	0.7292	0.0000
0.9894	33.0	495	0.9643	0.7188	0.0000
0.9503	34.0	510	0.9531	0.7292	0.0000
0.9382	35.0	525	0.9386	0.7188	0.0000
0.9371	36.0	540	0.9313	0.7188	4e-05
0.9207	37.0	555	0.9192	0.7188	0.0000
0.9422	38.0	570	0.9091	0.7292	0.0000
0.9041	39.0	585	0.8975	0.7188	0.0000
0.8966	40.0	600	0.8888	0.7188	0.0000
0.8877	41.0	615	0.8800	0.75	0.0000
0.8933	42.0	630	0.8748	0.7396	3e-05
0.854	43.0	645	0.8629	0.75	0.0000
0.8858	44.0	660	0.8580	0.7396	0.0000
0.8407	45.0	675	0.8516	0.75	0.0000
0.8413	46.0	690	0.8475	0.75	0.0000
0.86	47.0	705	0.8416	0.7396	0.0000
0.8282	48.0	720	0.8332	0.7396	2e-05
0.835	49.0	735	0.8303	0.7396	0.0000
0.8123	50.0	750	0.8267	0.7396	0.0000
0.8175	51.0	765	0.8227	0.7396	0.0000
0.8097	52.0	780	0.8177	0.75	0.0000
0.806	53.0	795	0.8155	0.7396	0.0000

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.1