neptun-org/neptun.ai

Neptun AI

RWKV-5-World-0.4B-v2-20231113-ctx4096.pth tuned using neptun.scraper and LoRA Finetune.
Converted using convert_ai00.py.

• Precision: bf16
• Context Length: 4096
• Epoch Begin: 0
• Epoch Count: 20
• Epoch Steps: 1000
• Epoch Save: 1000
• Warmup Steps: 0
• Learning Rate Init: 2e-5
• Learning Rate Final: 2e-5
• Micro Batch Size: 1
• Accumulate Gradient Batches: 24
• Beta 1: 0,9
• Beta 2: 0,999
• LoRA R: 64
• LoRA Alpha: 192
• LoRA Dropout: 0,01
• Adam Epsilon: 1e-8
• Gradient Checkpoint: true
• Devices: 1

Dockerfiles

1 1.552413 4.7229 0.00002000 2025-01-28 14:47:38.809119 0
2 1.478129 4.3847 0.00002000 2025-01-28 15:37:13.198567 1
3 1.412234 4.1051 0.00002000 2025-01-28 16:26:29.863755 2
4 1.374447 3.9529 0.00002000 2025-01-28 17:16:15.815151 3
5 1.317078 3.7325 0.00002000 2025-01-28 18:06:02.229190 4
6 1.296596 3.6568 0.00002000 2025-01-28 18:55:40.765344 5
7 1.257023 3.5149 0.00002000 2025-01-28 19:45:23.963291 6
8 1.253057 3.5010 0.00002000 2025-01-28 20:34:39.364658 7
9 1.219242 3.3846 0.00002000 2025-01-28 21:23:37.893528 8
10 1.190037 3.2872 0.00002000 2025-01-28 22:12:28.025686 9
11 1.181473 3.2592 0.00002000 2025-01-28 23:02:49.004392 10
12 1.176299 3.2424 0.00002000 2025-01-28 23:52:07.349197 11
13 1.153920 3.1706 0.00002000 2025-01-29 00:42:01.340925 12
14 1.146656 3.1477 0.00002000 2025-01-29 01:30:47.291425 13
15 1.125227 3.0809 0.00002000 2025-01-29 02:19:49.021365 14
16 1.125680 3.0823 0.00002000 2025-01-29 03:08:48.035252 15
17 1.108051 3.0284 0.00002000 2025-01-29 03:57:08.758836 16
18 1.121515 3.0695 0.00002000 2025-01-29 04:45:28.513407 17
19 1.098906 3.0009 0.00002000 2025-01-29 05:33:47.381810 18
20 1.088252 2.9691 0.00002000 2025-01-29 06:22:06.299740 19

V1

0 1.328085 3.7738 0.00002000 2025-02-24 22:42:19.385036 0
1 1.250953 3.4937 0.00002000 2025-02-24 23:31:09.470548 1
2 1.220156 3.3877 0.00002000 2025-02-25 00:20:28.265176 2
3 1.189695 3.2861 0.00002000 2025-02-25 01:09:00.092662 3
4 1.173590 3.2336 0.00002000 2025-02-25 01:57:14.064583 4
5 1.155840 3.1767 0.00002000 2025-02-25 02:45:25.223964 5
6 1.146094 3.1459 0.00002000 2025-02-25 03:33:37.876671 6
7 1.128762 3.0918 0.00002000 2025-02-25 04:21:51.325201 7
8 1.119391 3.0630 0.00002000 2025-02-25 05:10:04.734031 8
9 1.112895 3.0432 0.00002000 2025-02-25 05:58:17.806360 9
10 1.109004 3.0313 0.00002000 2025-02-25 06:46:33.949976 10
11 1.101230 3.0079 0.00002000 2025-02-25 07:34:52.654477 11
12 1.092156 2.9807 0.00002000 2025-02-25 08:23:09.908699 12
13 1.087285 2.9662 0.00002000 2025-02-25 09:11:25.518814 13
14 1.081148 2.9481 0.00002000 2025-02-25 09:59:37.800531 14
15 1.074223 2.9277 0.00002000 2025-02-25 10:47:51.388452 15
16 1.074324 2.9280 0.00002000 2025-02-25 11:36:07.807562 16
17 1.068234 2.9102 0.00002000 2025-02-25 12:24:22.401156 17
18 1.059605 2.8852 0.00002000 2025-02-25 13:12:37.599124 18
19 1.055234 2.8726 0.00002000 2025-02-25 14:00:52.741654 19

V2

1 1.432728 4.1901 0.00002000 2025-03-01 18:28:38.262673 0
2 1.295736 3.6537 0.00002000 2025-03-01 19:17:53.931170 1
3 1.277158 3.5864 0.00002000 2025-03-01 20:07:08.693268 2
4 1.266635 3.5489 0.00002000 2025-03-01 20:56:43.106751 3
5 1.232101 3.4284 0.00002000 2025-03-01 21:46:04.793249 4
6 1.197623 3.3122 0.00002000 2025-03-01 22:36:19.047485 5
7 1.198684 3.3157 0.00002000 2025-03-01 23:26:23.852325 6
8 1.153539 3.1694 0.00002000 2025-03-02 00:15:25.319388 7
9 1.190393 3.2884 0.00002000 2025-03-02 01:04:04.914936 8
10 1.151376 3.1625 0.00002000 2025-03-02 01:52:36.610501 9
11 1.142265 3.1339 0.00002000 2025-03-02 02:41:09.421198 10
12 1.125220 3.0809 0.00002000 2025-03-02 03:29:40.905325 11
13 1.108536 3.0299 0.00002000 2025-03-02 04:18:13.008727 12
14 1.152535 3.1662 0.00002000 2025-03-02 05:06:43.459698 13
15 1.137459 3.1188 0.00002000 2025-03-02 05:55:18.549682 14
16 1.107539 3.0269 0.00002000 2025-03-02 06:43:50.482294 15
17 1.137236 3.1181 0.00002000 2025-03-02 07:32:25.326706 16
18 1.119796 3.0642 0.00002000 2025-03-02 08:20:57.625010 17
19 1.109311 3.0323 0.00002000 2025-03-02 09:09:29.684893 18
20 1.098197 2.9988 0.00002000 2025-03-02 09:58:02.055555 19