KannadaBERT-lamb

This model is a fine-tuned version of Chakita/KannadaBERT on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.2098

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 200

Training results

Training Loss	Epoch	Step	Validation Loss
No log	1.0	319	3.6259
3.9523	2.0	638	3.6620
3.9523	3.0	957	3.5293
3.7334	4.0	1276	3.6279
3.6238	5.0	1595	3.4369
3.6238	6.0	1914	3.4624
3.5668	7.0	2233	3.3384
3.5026	8.0	2552	3.4141
3.5026	9.0	2871	3.3134
3.4171	10.0	3190	3.2900
3.358	11.0	3509	3.1716
3.358	12.0	3828	3.1200
3.2705	13.0	4147	3.1204
3.2705	14.0	4466	3.0478
3.1964	15.0	4785	2.9918
3.1388	16.0	5104	3.0790
3.1388	17.0	5423	2.9753
3.094	18.0	5742	2.9102
3.0608	19.0	6061	2.8584
3.0608	20.0	6380	2.7935
3.0029	21.0	6699	2.7997
2.9807	22.0	7018	2.8857
2.9807	23.0	7337	2.8606
2.9287	24.0	7656	2.8186
2.9287	25.0	7975	2.7788
2.894	26.0	8294	2.7850
2.8592	27.0	8613	2.6883
2.8592	28.0	8932	2.7090
2.8158	29.0	9251	2.5496
2.7705	30.0	9570	2.7173
2.7705	31.0	9889	2.5792
2.7425	32.0	10208	2.5922
2.7014	33.0	10527	2.5099
2.7014	34.0	10846	2.5285
2.6643	35.0	11165	2.5281
2.6643	36.0	11484	2.4245
2.6002	37.0	11803	2.4300
2.5816	38.0	12122	2.3803
2.5816	39.0	12441	2.2476
2.5257	40.0	12760	2.2816
2.5016	41.0	13079	2.2871
2.5016	42.0	13398	2.2967
2.4552	43.0	13717	2.2689
2.4262	44.0	14036	2.2345
2.4262	45.0	14355	2.3149
2.3905	46.0	14674	2.2341
2.3905	47.0	14993	2.1906
2.3551	48.0	15312	2.2131
2.3406	49.0	15631	2.1505
2.3406	50.0	15950	2.1251
2.3076	51.0	16269	2.1266
2.2673	52.0	16588	2.1688
2.2673	53.0	16907	2.0224
2.249	54.0	17226	2.0511
2.2314	55.0	17545	2.0409
2.2314	56.0	17864	2.0156
2.1968	57.0	18183	2.0767
2.1638	58.0	18502	1.9530
2.1638	59.0	18821	2.0355
2.1452	60.0	19140	1.9454
2.1452	61.0	19459	1.9321
2.1209	62.0	19778	1.9333
2.0946	63.0	20097	1.8804
2.0946	64.0	20416	1.9018
2.074	65.0	20735	1.8112
2.0394	66.0	21054	1.8049
2.0394	67.0	21373	1.8753
2.0188	68.0	21692	1.7983
1.99	69.0	22011	1.8464
1.99	70.0	22330	1.7434
1.9644	71.0	22649	1.7874
1.9644	72.0	22968	1.7875
1.9533	73.0	23287	1.8250
1.926	74.0	23606	1.7371
1.926	75.0	23925	1.7980
1.9171	76.0	24244	1.7354
1.8955	77.0	24563	1.8076
1.8955	78.0	24882	1.6711
1.878	79.0	25201	1.6909
1.8386	80.0	25520	1.6858
1.8386	81.0	25839	1.7103
1.837	82.0	26158	1.6797
1.837	83.0	26477	1.6439
1.8287	84.0	26796	1.6634
1.7927	85.0	27115	1.5949
1.7927	86.0	27434	1.6433
1.77	87.0	27753	1.6567
1.7684	88.0	28072	1.5484
1.7684	89.0	28391	1.6145
1.7423	90.0	28710	1.6318
1.7295	91.0	29029	1.5427
1.7295	92.0	29348	1.6212
1.7098	93.0	29667	1.5626
1.7098	94.0	29986	1.6377
1.7087	95.0	30305	1.5701
1.6817	96.0	30624	1.4842
1.6817	97.0	30943	1.5769
1.6732	98.0	31262	1.5541
1.6512	99.0	31581	1.5648
1.6512	100.0	31900	1.5420
1.642	101.0	32219	1.4468
1.63	102.0	32538	1.4316
1.63	103.0	32857	1.4182
1.6254	104.0	33176	1.5147
1.6254	105.0	33495	1.5259
1.5967	106.0	33814	1.4105
1.5842	107.0	34133	1.3901
1.5842	108.0	34452	1.4855
1.5953	109.0	34771	1.5122
1.5772	110.0	35090	1.4486
1.5772	111.0	35409	1.3851
1.5643	112.0	35728	1.4139
1.5483	113.0	36047	1.3437
1.5483	114.0	36366	1.3224
1.5393	115.0	36685	1.3664
1.5156	116.0	37004	1.3576
1.5156	117.0	37323	1.4949
1.5242	118.0	37642	1.4403
1.5242	119.0	37961	1.3988
1.5263	120.0	38280	1.3910
1.5098	121.0	38599	1.4632
1.5098	122.0	38918	1.3871
1.4922	123.0	39237	1.3946
1.4898	124.0	39556	1.4390
1.4898	125.0	39875	1.2734
1.4807	126.0	40194	1.3209
1.4693	127.0	40513	1.3928
1.4693	128.0	40832	1.4490
1.466	129.0	41151	1.3710
1.466	130.0	41470	1.3957
1.4701	131.0	41789	1.2389
1.4411	132.0	42108	1.4310
1.4411	133.0	42427	1.3077
1.4374	134.0	42746	1.3514
1.4348	135.0	43065	1.3392
1.4348	136.0	43384	1.3673
1.4252	137.0	43703	1.3510
1.4209	138.0	44022	1.3027
1.4209	139.0	44341	1.2340
1.4195	140.0	44660	1.3203
1.4195	141.0	44979	1.3199
1.4001	142.0	45298	1.3095
1.416	143.0	45617	1.3576
1.416	144.0	45936	1.2701
1.4063	145.0	46255	1.2958
1.397	146.0	46574	1.2620
1.397	147.0	46893	1.2759
1.3987	148.0	47212	1.3024
1.374	149.0	47531	1.2234
1.374	150.0	47850	1.2767
1.3783	151.0	48169	1.2659
1.3783	152.0	48488	1.2980
1.3747	153.0	48807	1.3125
1.3761	154.0	49126	1.2958
1.3761	155.0	49445	1.3287
1.3605	156.0	49764	1.2724
1.3588	157.0	50083	1.3302
1.3588	158.0	50402	1.3242
1.352	159.0	50721	1.2390
1.3545	160.0	51040	1.2435
1.3545	161.0	51359	1.3246
1.3501	162.0	51678	1.2723
1.3501	163.0	51997	1.2139
1.3362	164.0	52316	1.2435
1.3457	165.0	52635	1.2198
1.3457	166.0	52954	1.2691
1.3298	167.0	53273	1.2945
1.335	168.0	53592	1.1958
1.335	169.0	53911	1.2011
1.3302	170.0	54230	1.3094
1.3258	171.0	54549	1.2617
1.3258	172.0	54868	1.3582
1.3151	173.0	55187	1.2570
1.3336	174.0	55506	1.2160
1.3336	175.0	55825	1.2224
1.3131	176.0	56144	1.2542
1.3131	177.0	56463	1.2518
1.3114	178.0	56782	1.2280
1.3034	179.0	57101	1.3342
1.3034	180.0	57420	1.2120
1.3124	181.0	57739	1.2109
1.3163	182.0	58058	1.2482
1.3163	183.0	58377	1.1576
1.3023	184.0	58696	1.2120
1.3135	185.0	59015	1.2895
1.3135	186.0	59334	1.2604
1.3053	187.0	59653	1.2423
1.3053	188.0	59972	1.1938
1.2964	189.0	60291	1.2544
1.2945	190.0	60610	1.2014
1.2945	191.0	60929	1.2223
1.2971	192.0	61248	1.2778
1.3007	193.0	61567	1.2427
1.3007	194.0	61886	1.2390
1.2885	195.0	62205	1.2286
1.3103	196.0	62524	1.2421
1.3103	197.0	62843	1.2015
1.3012	198.0	63162	1.2083
1.3012	199.0	63481	1.2305
1.2907	200.0	63800	1.1906

Framework versions

• Transformers 4.28.0
• Pytorch 2.0.1+cu117
• Datasets 2.12.0
• Tokenizers 0.13.2