OpenAI_Clip_Roco_ALL_V1

This model was trained from scratch on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.2898

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 2
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 5
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss
1.0986	0.0449	1000	2.7726
1.1005	0.0899	2000	3.4652
1.0327	0.1348	3000	3.3699
1.0075	0.1797	4000	3.3391
0.9483	0.2246	5000	3.2981
0.9581	0.2696	6000	3.3342
0.9475	0.3145	7000	3.2390
0.9405	0.3594	8000	3.2371
0.9075	0.4043	9000	3.1723
0.8889	0.4493	10000	3.1768
0.8537	0.4942	11000	3.1828
0.8321	0.5391	12000	3.1766
0.8757	0.5841	13000	3.1685
0.8147	0.6290	14000	3.1354
0.8132	0.6739	15000	3.0864
0.8218	0.7188	16000	3.1094
0.8082	0.7638	17000	3.0615
0.8065	0.8087	18000	2.9708
0.7344	0.8536	19000	2.9754
0.8059	0.8986	20000	2.9851
0.7412	0.9435	21000	2.9331
0.7751	0.9884	22000	3.0652
0.7491	1.0333	23000	3.0196
0.7081	1.0783	24000	3.0186
0.7292	1.1232	25000	2.8924
0.7304	1.1681	26000	2.9324
0.7092	1.2130	27000	2.9921
0.7055	1.2580	28000	2.9535
0.7292	1.3029	29000	2.9846
0.7234	1.3478	30000	2.9456
0.7554	1.3928	31000	2.9418
0.6696	1.4377	32000	2.9881
0.6928	1.4826	33000	2.9008
0.6702	1.5275	34000	2.8690
0.6826	1.5725	35000	2.9143
0.6964	1.6174	36000	2.8567
0.6872	1.6623	37000	2.9428
0.7538	1.7073	38000	2.8243
0.6908	1.7522	39000	2.8800
0.6627	1.7971	40000	2.9341
0.698	1.8420	41000	2.8495
0.6641	1.8870	42000	2.8374
0.6746	1.9319	43000	2.8288
0.7073	1.9768	44000	2.8696
0.6427	2.0217	45000	2.8303
0.6951	2.0667	46000	2.9055
0.6432	2.1116	47000	2.8455
0.6596	2.1565	48000	2.8067
0.6828	2.2015	49000	2.7990
0.6115	2.2464	50000	2.9128
0.6628	2.2913	51000	2.8097
0.6776	2.3362	52000	2.8564
0.6435	2.3812	53000	2.8078
0.6299	2.4261	54000	2.8412
0.6439	2.4710	55000	2.9027
0.6441	2.5159	56000	2.8362
0.6574	2.5609	57000	2.7768
0.6232	2.6058	58000	2.8551
0.6708	2.6507	59000	2.7753
0.6308	2.6957	60000	2.7702
0.6426	2.7406	61000	2.7874
0.6209	2.7855	62000	2.7115
0.6175	2.8304	63000	2.7555
0.656	2.8754	64000	2.7700
0.6257	2.9203	65000	2.7837
0.6157	2.9652	66000	2.8421
0.5943	3.0102	67000	2.7818
0.6033	3.0551	68000	2.8714
0.6176	3.1000	69000	2.7742
0.6235	3.1449	70000	2.7178
0.5612	3.1899	71000	2.6945
0.6139	3.2348	72000	2.7094
0.6381	3.2797	73000	2.7967
0.6147	3.3246	74000	2.8350
0.6118	3.3696	75000	2.7840
0.5684	3.4145	76000	2.7369
0.582	3.4594	77000	2.7216
0.6236	3.5044	78000	2.6894
0.6256	3.5493	79000	2.7475
0.5926	3.5942	80000	2.7006
0.5875	3.6391	81000	2.7505
0.6267	3.6841	82000	2.8129
0.6231	3.7290	83000	2.6899
0.5944	3.7739	84000	2.7181
0.5849	3.8189	85000	2.7390
0.5848	3.8638	86000	2.7292
0.6302	3.9087	87000	2.6868
0.5873	3.9536	88000	2.6699
0.6081	3.9986	89000	2.7178
0.5711	4.0435	90000	2.7641
0.5665	4.0884	91000	2.6880
0.5902	4.1333	92000	2.6802
0.5496	4.1783	93000	2.7101
0.5953	4.2232	94000	2.7122
0.6262	4.2681	95000	2.6548
0.5867	4.3131	96000	2.6919
0.6085	4.3580	97000	2.6769
0.5692	4.4029	98000	2.7150
0.5542	4.4478	99000	2.6833
0.5978	4.4928	100000	2.6962
0.5933	4.5377	101000	2.7136
0.5363	4.5826	102000	2.7156
0.6273	4.6275	103000	2.6751
0.5461	4.6725	104000	2.6609
0.5901	4.7174	105000	2.6357
0.5926	4.7623	106000	2.6782
0.572	4.8073	107000	2.6387
0.585	4.8522	108000	2.6704
0.5684	4.8971	109000	2.6129
0.5587	4.9420	110000	2.6792
0.5718	4.9870	111000	2.6825
0.5168	5.0319	112000	2.6722
0.5399	5.0768	113000	2.6921
0.549	5.1218	114000	2.6621
0.5792	5.1667	115000	2.6696
0.5843	5.2116	116000	2.7064
0.607	5.2565	117000	2.6801
0.562	5.3015	118000	2.6795
0.564	5.3464	119000	2.6711
0.5814	5.3913	120000	2.5865
0.5389	5.4362	121000	2.6354
0.5023	5.4812	122000	2.6258
0.5343	5.5261	123000	2.6320
0.5379	5.5710	124000	2.6876
0.521	5.6160	125000	2.6383
0.5351	5.6609	126000	2.6288
0.5791	5.7058	127000	2.6114
0.5535	5.7507	128000	2.6783
0.5144	5.7957	129000	2.6545
0.5761	5.8406	130000	2.5853
0.5808	5.8855	131000	2.6103
0.5903	5.9305	132000	2.6288
0.5342	5.9754	133000	2.6159
0.553	6.0203	134000	2.6343
0.5413	6.0652	135000	2.6129
0.5798	6.1102	136000	2.6443
0.5296	6.1551	137000	2.6668
0.516	6.2000	138000	2.5899
0.5327	6.2449	139000	2.6326
0.5322	6.2899	140000	2.6203
0.5473	6.3348	141000	2.6474
0.5504	6.3797	142000	2.6440
0.5271	6.4247	143000	2.6617
0.5278	6.4696	144000	2.5856
0.5455	6.5145	145000	2.6387
0.4836	6.5594	146000	2.7035
0.5519	6.6044	147000	2.6106
0.5514	6.6493	148000	2.6822
0.5331	6.6942	149000	2.5944
0.5371	6.7391	150000	2.5755
0.544	6.7841	151000	2.5801
0.535	6.8290	152000	2.6218
0.5423	6.8739	153000	2.6196
0.5204	6.9189	154000	2.6578
0.5732	6.9638	155000	2.6092
0.4809	7.0087	156000	2.6050
0.511	7.0536	157000	2.6104
0.5387	7.0986	158000	2.6241
0.4832	7.1435	159000	2.6097
0.4882	7.1884	160000	2.5921
0.5099	7.2334	161000	2.6301
0.5518	7.2783	162000	2.5808
0.5447	7.3232	163000	2.5550
0.5287	7.3681	164000	2.5781
0.5514	7.4131	165000	2.5828
0.5135	7.4580	166000	2.5978
0.5371	7.5029	167000	2.5369
0.5153	7.5478	168000	2.6238
0.5109	7.5928	169000	2.5884
0.4822	7.6377	170000	2.6600
0.5241	7.6826	171000	2.5465
0.4941	7.7276	172000	2.6000
0.5042	7.7725	173000	2.6264
0.5106	7.8174	174000	2.6233
0.5052	7.8623	175000	2.5763
0.5266	7.9073	176000	2.5732
0.5361	7.9522	177000	2.5877
0.5193	7.9971	178000	2.6212
0.475	8.0421	179000	2.6140
0.5183	8.0870	180000	2.6353
0.5076	8.1319	181000	2.5748
0.4791	8.1768	182000	2.6219
0.4812	8.2218	183000	2.6147
0.4801	8.2667	184000	2.6716
0.4898	8.3116	185000	2.5693
0.5183	8.3565	186000	2.5466
0.5324	8.4015	187000	2.5965
0.4847	8.4464	188000	2.5656
0.5246	8.4913	189000	2.5952
0.4905	8.5363	190000	2.6485
0.4862	8.5812	191000	2.6468
0.5209	8.6261	192000	2.5381
0.5135	8.6710	193000	2.6620
0.493	8.7160	194000	2.6196
0.491	8.7609	195000	2.5628
0.4984	8.8058	196000	2.6433
0.4943	8.8508	197000	2.5749
0.5205	8.8957	198000	2.5894
0.459	8.9406	199000	2.5856
0.51	8.9855	200000	2.5565
0.4924	9.0305	201000	2.6095
0.5253	9.0754	202000	2.5797
0.5063	9.1203	203000	2.5658
0.4851	9.1652	204000	2.5868
0.4967	9.2102	205000	2.5801
0.5259	9.2551	206000	2.6047
0.503	9.3000	207000	2.5673
0.4998	9.3450	208000	2.5886
0.5253	9.3899	209000	2.6000
0.5043	9.4348	210000	2.5735
0.5008	9.4797	211000	2.5436
0.5192	9.5247	212000	2.5792
0.4759	9.5696	213000	2.6342
0.4777	9.6145	214000	2.5285
0.5264	9.6594	215000	2.6182
0.5262	9.7044	216000	2.5615
0.4631	9.7493	217000	2.5767
0.447	9.7942	218000	2.6014
0.4505	9.8392	219000	2.5184
0.4863	9.8841	220000	2.5282
0.5231	9.9290	221000	2.5266
0.4859	9.9739	222000	2.5313
0.4717	10.0189	223000	2.5421
0.4674	10.0638	224000	2.5632
0.4777	10.1087	225000	2.5895
0.4963	10.1537	226000	2.5650
0.446	10.1986	227000	2.5394
0.4512	10.2435	228000	2.5607
0.4893	10.2884	229000	2.5368
0.4586	10.3334	230000	2.5846
0.4565	10.3783	231000	2.5332
0.4789	10.4232	232000	2.5550
0.4652	10.4681	233000	2.5368
0.5054	10.5131	234000	2.5644
0.4384	10.5580	235000	2.5507
0.449	10.6029	236000	2.5376
0.4704	10.6479	237000	2.5733
0.5095	10.6928	238000	2.5306
0.4882	10.7377	239000	2.5735
0.5051	10.7826	240000	2.6252
0.4912	10.8276	241000	2.5472
0.464	10.8725	242000	2.5376
0.5158	10.9174	243000	2.5293
0.4819	10.9624	244000	2.5245
0.4478	11.0073	245000	2.5267
0.4706	11.0522	246000	2.5728
0.4627	11.0971	247000	2.5437
0.4675	11.1421	248000	2.5572
0.4923	11.1870	249000	2.5094
0.4676	11.2319	250000	2.5488
0.4921	11.2768	251000	2.6066
0.4654	11.3218	252000	2.5903
0.4602	11.3667	253000	2.5286
0.4634	11.4116	254000	2.5327
0.4588	11.4566	255000	2.5937
0.483	11.5015	256000	2.5679
0.4649	11.5464	257000	2.5158
0.4709	11.5913	258000	2.6008
0.4771	11.6363	259000	2.5678
0.4671	11.6812	260000	2.5642
0.4535	11.7261	261000	2.5357
0.4235	11.7710	262000	2.5822
0.4757	11.8160	263000	2.5642
0.4688	11.8609	264000	2.9452
0.5091	11.9058	265000	2.9430
0.4642	11.9508	266000	3.0008
0.4415	11.9957	267000	2.9907
0.4374	12.0406	268000	2.9819
0.4056	12.0855	269000	2.9604
0.42	12.1305	270000	3.0124
0.4497	12.1754	271000	2.9802
0.4471	12.2203	272000	2.9552
0.5112	12.2653	273000	2.9935
0.4582	12.3102	274000	3.0678
0.4361	12.3551	275000	2.9659
0.4493	12.4000	276000	3.9247
0.4306	12.4450	277000	3.9121
0.4549	12.4899	278000	3.8761
0.4485	12.5348	279000	3.8943
0.4969	12.5797	280000	3.8590
0.4898	12.6247	281000	3.8365
0.4835	12.6696	282000	3.9126
0.4452	12.7145	283000	3.8461
0.4957	12.7595	284000	3.8401
0.4713	12.8044	285000	3.8345
0.4586	12.8493	286000	3.8653
0.4433	12.8942	287000	3.8534
0.4955	12.9392	288000	3.7864
0.4632	12.9841	289000	3.8833
0.459	13.0290	290000	3.8505
0.4483	13.0740	291000	3.8966
0.4664	13.1189	292000	3.8597
0.4243	13.1638	293000	3.8728
0.3967	13.2087	294000	3.8817
0.4462	13.2537	295000	3.8484
0.4109	13.2986	296000	3.9298
0.4535	13.3435	297000	3.8182
0.4563	13.3884	298000	3.8600
0.4754	13.4334	299000	3.8714
0.4103	13.4783	300000	3.9464
0.4347	13.5232	301000	3.8569
0.4566	13.5682	302000	3.8379
0.4471	13.6131	303000	3.8606
0.4752	13.6580	304000	3.8503
0.4484	13.7029	305000	3.8592
0.451	13.7479	306000	3.9124
0.4948	13.7928	307000	3.8661
0.4393	13.8377	308000	3.8695
0.4455	13.8826	309000	3.8585
0.4334	13.9276	310000	3.8456
0.448	13.9725	311000	3.8607
0.3854	14.0174	312000	3.9007
0.4323	14.0624	313000	3.8580
0.4153	14.1073	314000	3.8607
0.4383	14.1522	315000	3.8739
0.4476	14.1971	316000	3.8536
0.3815	14.2421	317000	3.8721
0.4184	14.2870	318000	3.8637
0.4277	14.3319	319000	3.8313
0.4581	14.3769	320000	3.7998
0.4677	14.4218	321000	3.8351
0.4171	14.4667	322000	3.8441
0.4367	14.5116	323000	3.8401
0.4572	14.5566	324000	3.8130
0.4811	14.6015	325000	3.8442
0.4476	14.6464	326000	3.8490
0.5084	14.6913	327000	3.8497
0.4241	14.7363	328000	3.9515
0.4596	14.7812	329000	3.8449
0.4871	14.8261	330000	3.8736
0.4313	14.8711	331000	3.8447
0.4698	14.9160	332000	3.8392
0.4158	14.9609	333000	3.8127
0.386	15.0058	334000	3.8316
0.4114	15.0508	335000	3.8605
0.4061	15.0957	336000	3.8789
0.4076	15.1406	337000	3.8616
0.3919	15.1856	338000	3.9069
0.4575	15.2305	339000	3.8878
0.4167	15.2754	340000	3.8731
0.4006	15.3203	341000	3.9192
0.4045	15.3653	342000	3.8500
0.4416	15.4102	343000	3.8163
0.4088	15.4551	344000	3.8348
0.3982	15.5000	345000	3.8165
0.4381	15.5450	346000	3.8208
0.3956	15.5899	347000	3.8776
0.4411	15.6348	348000	3.8424
0.4112	15.6798	349000	3.8427
0.4078	15.7247	350000	3.8592
0.4446	15.7696	351000	3.8350
0.4295	15.8145	352000	3.8360
0.434	15.8595	353000	3.8414
0.43	15.9044	354000	3.8792
0.4601	15.9493	355000	3.8631
0.4137	15.9942	356000	3.8049
0.4276	16.0392	357000	3.8480
0.3763	16.0841	358000	3.8912
0.436	16.1290	359000	3.8459
0.4164	16.1740	360000	3.8807
0.4133	16.2189	361000	3.8468
0.4021	16.2638	362000	3.8530
0.39	16.3087	363000	3.8928
0.4109	16.3537	364000	3.8154
0.4001	16.3986	365000	3.8781
0.3897	16.4435	366000	3.8389
0.4218	16.4885	367000	3.8613
0.4137	16.5334	368000	3.8438
0.4116	16.5783	369000	3.8293
0.4082	16.6232	370000	3.8366
0.4386	16.6682	371000	3.8591
0.405	16.7131	372000	3.8738
0.4134	16.7580	373000	3.9515
0.4258	16.8029	374000	3.8408
0.4242	16.8479	375000	3.9013
0.4451	16.8928	376000	3.8091
0.3846	16.9377	377000	3.8464
0.3805	16.9827	378000	3.9429
0.3912	17.0276	379000	3.8199
0.3701	17.0725	380000	3.8478
0.3727	17.1174	381000	3.8517
0.3988	17.1624	382000	3.8172
0.4363	17.2073	383000	3.8187
0.3812	17.2522	384000	3.9134
0.3557	17.2972	385000	3.8853
0.4209	17.3421	386000	3.8043
0.3743	17.3870	387000	3.8549
0.3763	17.4319	388000	3.8110
0.4258	17.4769	389000	3.8038
0.3711	17.5218	390000	3.8192
0.4094	17.5667	391000	3.8144
0.401	17.6116	392000	3.7937
0.4056	17.6566	393000	3.8447
0.3927	17.7015	394000	3.8253
0.3833	17.7464	395000	3.8368
0.3653	17.7914	396000	3.8246
0.4154	17.8363	397000	3.8302
0.3958	17.8812	398000	3.8028
0.392	17.9261	399000	3.7848
0.4061	17.9711	400000	3.8022
0.3737	18.0160	401000	3.8542
0.3402	18.0609	402000	3.8658
0.3648	18.1058	403000	3.8671
0.3513	18.1508	404000	3.8259
0.3896	18.1957	405000	3.8315
0.3678	18.2406	406000	3.8045
0.3554	18.2856	407000	3.9155
0.4102	18.3305	408000	3.8599
0.3875	18.3754	409000	3.8313
0.3745	18.4203	410000	3.8476
0.3916	18.4653	411000	3.8483
0.3713	18.5102	412000	3.8872
0.3525	18.5551	413000	3.7986
0.4015	18.6001	414000	3.8459
0.3608	18.6450	415000	3.8545
0.4236	18.6899	416000	3.7908
0.4017	18.7348	417000	3.8190
0.4224	18.7798	418000	3.8412
0.38	18.8247	419000	3.8336
0.3631	18.8696	420000	3.8032
0.383	18.9145	421000	3.8579
0.383	18.9595	422000	3.8788
0.405	19.0044	423000	3.8710
0.3385	19.0493	424000	3.8625
0.393	19.0943	425000	3.8475
0.3514	19.1392	426000	3.8293
0.3502	19.1841	427000	3.8812
0.3875	19.2290	428000	3.8519
0.3349	19.2740	429000	3.8657
0.4075	19.3189	430000	3.8725
0.3454	19.3638	431000	3.8316
0.398	19.4088	432000	3.8530
0.3864	19.4537	433000	3.8706
0.3916	19.4986	434000	3.8529
0.3445	19.5435	435000	3.9160
0.3816	19.5885	436000	3.8427
0.3735	19.6334	437000	3.8144
0.4164	19.6783	438000	3.8025
0.3778	19.7232	439000	3.8162
0.401	19.7682	440000	3.8067
0.35	19.8131	441000	3.8280
0.3936	19.8580	442000	3.8562
0.3749	19.9030	443000	3.8550
0.4003	19.9479	444000	3.7951
0.4145	19.9928	445000	3.7546
0.3828	20.0377	446000	3.8768
0.3559	20.0827	447000	3.8682
0.3359	20.1276	448000	3.8780
0.3382	20.1725	449000	3.8710
0.3804	20.2174	450000	3.9251
0.3624	20.2624	451000	3.8094
0.377	20.3073	452000	3.8351
0.3757	20.3522	453000	3.8324
0.3704	20.3972	454000	3.8862
0.37	20.4421	455000	3.8735
0.3457	20.4870	456000	3.8304
0.3422	20.5319	457000	3.8799
0.3513	20.5769	458000	3.8579
0.3904	20.6218	459000	3.8455
0.3311	20.6667	460000	3.8991
0.3855	20.7117	461000	3.8244
0.3833	20.7566	462000	3.8414
0.3742	20.8015	463000	3.8070
0.3761	20.8464	464000	3.8668
0.3897	20.8914	465000	3.8309
0.3539	20.9363	466000	3.8358
0.3255	20.9812	467000	3.8359
0.2882	21.0261	468000	3.8817
0.329	21.0711	469000	3.8374
0.3399	21.1160	470000	3.8397
0.3404	21.1609	471000	3.8528
0.3651	21.2059	472000	3.8473
0.3848	21.2508	473000	3.8795
0.3606	21.2957	474000	3.8442
0.3567	21.3406	475000	3.8480
0.4052	21.3856	476000	3.8101
0.3432	21.4305	477000	3.8364
0.3906	21.4754	478000	3.8568
0.3594	21.5204	479000	3.8206
0.3689	21.5653	480000	3.8755
0.3595	21.6102	481000	3.8320
0.3254	21.6551	482000	3.8365
0.3684	21.7001	483000	3.8798
0.3347	21.7450	484000	3.8593
0.3624	21.7899	485000	3.8206
0.3698	21.8348	486000	3.7959
0.3484	21.8798	487000	3.8458
0.3934	21.9247	488000	3.8679
0.3349	21.9696	489000	3.8698
0.3381	22.0146	490000	3.8692
0.3167	22.0595	491000	3.9672
0.3324	22.1044	492000	3.9013
0.3305	22.1493	493000	3.8642
0.3182	22.1943	494000	3.8617
0.3633	22.2392	495000	3.8879
0.3282	22.2841	496000	3.8440
0.4121	29.7712	497000	1.3002
0.4185	29.8311	498000	1.3007
0.4	29.8910	499000	1.2910
0.4009	29.9509	500000	1.2899
0.3744	30.0108	501000	1.2981
0.3691	30.0707	502000	1.2914
0.3775	30.1306	503000	1.2877
0.3604	30.1905	504000	1.2967
0.3934	30.2504	505000	1.2898

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1