camembert_ccnet_classification_tools_fr_V2

This model is a fine-tuned version of camembert/camembert-base-ccnet on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3572
• Accuracy: 0.9479
• Learning Rate: 0.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 24
• eval_batch_size: 192
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 60

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Rate
1.7727	1.0	15	1.2524	0.7708	0.0001
0.9027	2.0	30	0.6733	0.8646	0.0001
0.4256	3.0	45	0.4453	0.8646	0.0001
0.2468	4.0	60	0.2961	0.9375	0.0001
0.1407	5.0	75	0.2918	0.9271	0.0001
0.082	6.0	90	0.3904	0.9062	9e-05
0.0486	7.0	105	0.2854	0.9479	0.0001
0.0821	8.0	120	0.2282	0.9479	0.0001
0.044	9.0	135	0.2699	0.9375	0.0001
0.0308	10.0	150	0.2305	0.9479	0.0001
0.0201	11.0	165	0.3650	0.9271	0.0001
0.0106	12.0	180	0.3281	0.9479	8e-05
0.0079	13.0	195	0.3435	0.9479	0.0001
0.007	14.0	210	0.3455	0.9479	0.0001
0.0322	15.0	225	0.2507	0.9583	0.0001
0.0055	16.0	240	0.2874	0.9583	0.0001
0.0047	17.0	255	0.2885	0.9583	0.0001
0.0042	18.0	270	0.2875	0.9583	7e-05
0.0039	19.0	285	0.2922	0.9583	0.0001
0.0035	20.0	300	0.2968	0.9583	0.0001
0.0033	21.0	315	0.3038	0.9479	0.0001
0.0031	22.0	330	0.3108	0.9479	0.0001
0.0029	23.0	345	0.3141	0.9479	0.0001
0.0027	24.0	360	0.3151	0.9479	6e-05
0.0025	25.0	375	0.3170	0.9479	0.0001
0.0023	26.0	390	0.3186	0.9479	0.0001
0.0023	27.0	405	0.3194	0.9479	0.0001
0.0022	28.0	420	0.3213	0.9479	0.0001
0.0021	29.0	435	0.3238	0.9479	0.0001
0.002	30.0	450	0.3246	0.9479	5e-05
0.0019	31.0	465	0.3273	0.9479	0.0000
0.0018	32.0	480	0.3284	0.9479	0.0000
0.0017	33.0	495	0.3383	0.9479	0.0000
0.0017	34.0	510	0.3408	0.9479	0.0000
0.0017	35.0	525	0.3423	0.9479	0.0000
0.0016	36.0	540	0.3436	0.9479	4e-05
0.0015	37.0	555	0.3451	0.9479	0.0000
0.0015	38.0	570	0.3459	0.9479	0.0000
0.0014	39.0	585	0.3467	0.9479	0.0000
0.0014	40.0	600	0.3479	0.9479	0.0000
0.0013	41.0	615	0.3488	0.9479	0.0000
0.0013	42.0	630	0.3502	0.9479	3e-05
0.0013	43.0	645	0.3501	0.9479	0.0000
0.0013	44.0	660	0.3499	0.9479	0.0000
0.0012	45.0	675	0.3506	0.9479	0.0000
0.0012	46.0	690	0.3515	0.9479	0.0000
0.0012	47.0	705	0.3522	0.9479	0.0000
0.0012	48.0	720	0.3528	0.9479	2e-05
0.0011	49.0	735	0.3534	0.9479	0.0000
0.0012	50.0	750	0.3541	0.9479	0.0000
0.0011	51.0	765	0.3545	0.9479	0.0000
0.0011	52.0	780	0.3549	0.9479	0.0000
0.0011	53.0	795	0.3556	0.9479	0.0000
0.0011	54.0	810	0.3560	0.9479	1e-05
0.0011	55.0	825	0.3563	0.9479	0.0000
0.0011	56.0	840	0.3567	0.9479	0.0000
0.0011	57.0	855	0.3570	0.9479	5e-06
0.0011	58.0	870	0.3571	0.9479	0.0000
0.0011	59.0	885	0.3572	0.9479	0.0000
0.001	60.0	900	0.3572	0.9479	0.0

Framework versions

• Transformers 4.34.0
• Pytorch 2.0.1+cu117
• Datasets 2.14.5
• Tokenizers 0.14.1