genrm-deepseek-ai-DeepSeek-R1-Distill-Qwen-14B

This model is a fine-tuned version of deepseek-ai/DeepSeek-R1-Distill-Qwen-14B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2953

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3e-07
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• total_train_batch_size: 8
• total_eval_batch_size: 8
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss
0.2028	0.0163	200	0.3168
0.1841	0.0326	400	0.3174
0.178	0.0489	600	0.3028
0.193	0.0652	800	0.3068
0.1912	0.0815	1000	0.3044
0.189	0.0979	1200	0.3004
0.1857	0.1142	1400	0.3047
0.175	0.1305	1600	0.3043
0.1717	0.1468	1800	0.2981
0.1779	0.1631	2000	0.3049
0.1727	0.1794	2200	0.3041
0.1796	0.1957	2400	0.3054
0.1843	0.2120	2600	0.2994
0.1832	0.2283	2800	0.3020
0.1642	0.2446	3000	0.3049
0.1954	0.2609	3200	0.2988
0.1884	0.2773	3400	0.2992
0.1717	0.2936	3600	0.2983
0.1744	0.3099	3800	0.2982
0.1717	0.3262	4000	0.3021
0.1519	0.3425	4200	0.2989
0.1857	0.3588	4400	0.2981
0.1802	0.3751	4600	0.3004
0.1637	0.3914	4800	0.2981
0.1611	0.4077	5000	0.2993
0.1957	0.4240	5200	0.2973
0.169	0.4403	5400	0.2950
0.1542	0.4567	5600	0.2972
0.1669	0.4730	5800	0.2943
0.1667	0.4893	6000	0.2944
0.1742	0.5056	6200	0.2963
0.1676	0.5219	6400	0.2951
0.1537	0.5382	6600	0.2975
0.1876	0.5545	6800	0.2979
0.1543	0.5708	7000	0.2980
0.1709	0.5871	7200	0.2981
0.1608	0.6034	7400	0.2967
0.1727	0.6198	7600	0.2973
0.1659	0.6361	7800	0.2959
0.1862	0.6524	8000	0.2990
0.1758	0.6687	8200	0.2974
0.1981	0.6850	8400	0.2965
0.1616	0.7013	8600	0.2943
0.1738	0.7176	8800	0.2943
0.1775	0.7339	9000	0.2958
0.1683	0.7502	9200	0.2949
0.1632	0.7665	9400	0.2946
0.1702	0.7828	9600	0.2937
0.1937	0.7992	9800	0.2944
0.162	0.8155	10000	0.2964
0.167	0.8318	10200	0.2968
0.1708	0.8481	10400	0.2963
0.16	0.8644	10600	0.2970
0.1695	0.8807	10800	0.2944
0.1568	0.8970	11000	0.2948
0.1708	0.9133	11200	0.2952
0.1561	0.9296	11400	0.2961
0.158	0.9459	11600	0.2950
0.1763	0.9622	11800	0.2948
0.1579	0.9786	12000	0.2950
0.1512	0.9949	12200	0.2953

Framework versions

• Transformers 4.46.1
• Pytorch 2.5.1+cu124
• Datasets 3.1.0
• Tokenizers 0.20.3