YuchenLi01 commited on
Commit
37afd34
·
verified ·
1 Parent(s): 5b79e73

Training in progress, step 664

Browse files
logs/amlt_code_runner.txt CHANGED
@@ -1,13 +1,13 @@
1
- 2025-04-16 14:50:59,197:amlt-code-runner:INFO - SINGULARITY_LOCATION: centralus
2
- 2025-04-16 14:50:59,197:amlt-code-runner:INFO - AISC_INSTANCE_TYPE: Singularity.ND96_v4
3
- 2025-04-16 14:51:02,324:amlt-code-runner:INFO - Not removing AzureML's cd commands from /etc/profile due to an error: [Errno 13] Permission denied: '/etc/profile'
4
- 2025-04-16 14:51:02,324:amlt-code-runner:WARNING - Environment variable 'NCCL_SOCKET_IFNAME' already set to '=eth0', not changing to '^docker0,lo'
5
- 2025-04-16 14:51:02,324:amlt-code-runner:INFO - RANK = 0
6
- 2025-04-16 14:51:02,324:amlt-code-runner:INFO - LOCAL_RANK = None
7
- 2025-04-16 14:51:02,324:amlt-code-runner:INFO - WORLD_SIZE = 1
8
- 2025-04-16 14:51:02,324:amlt-code-runner:INFO - MASTER_ADDR = node-0
9
- 2025-04-16 14:51:02,324:amlt-code-runner:INFO - MASTER_PORT = 9500
10
- 2025-04-16 14:51:02,325:amlt-code-runner:WARNING - Installing amlt runtime dependencies: ['wrapt', 'azure-identity', 'python-dateutil', 'pytz'] into /tmp/amlt-user-base
11
- 2025-04-16 14:51:03,958:amlt-code-runner:INFO - Executing ./amlt_setup.sh, ./amlt_run.sh
12
- 2025-04-16 14:51:04,031:background_dirsync:INFO - Starting directory syncer from '/scratch/amlt_code/outputs' to '/mnt/output/projects/amlt_project/amlt-results/7255445584.50642-958a35e9-1f0d-47f0-aae4-fca2fa07f65f', every 30.000000s
13
- 2025-04-16 14:51:04,034:background_dirsync:INFO - Starting directory syncer from '/scratch/azureml/cr/j/83f466416e734f4882203d7e05002400/exe/wd/logs' to '/scratch/amlt_code/outputs/logs', every 30.000000s
 
1
+ 2025-04-16 15:26:23,189:amlt-code-runner:INFO - SINGULARITY_LOCATION: centralus
2
+ 2025-04-16 15:26:23,189:amlt-code-runner:INFO - AISC_INSTANCE_TYPE: Singularity.ND96_v4
3
+ 2025-04-16 15:26:26,211:amlt-code-runner:INFO - Not removing AzureML's cd commands from /etc/profile due to an error: [Errno 13] Permission denied: '/etc/profile'
4
+ 2025-04-16 15:26:26,211:amlt-code-runner:WARNING - Environment variable 'NCCL_SOCKET_IFNAME' already set to '=eth0', not changing to '^docker0,lo'
5
+ 2025-04-16 15:26:26,211:amlt-code-runner:INFO - RANK = 0
6
+ 2025-04-16 15:26:26,211:amlt-code-runner:INFO - LOCAL_RANK = None
7
+ 2025-04-16 15:26:26,211:amlt-code-runner:INFO - WORLD_SIZE = 1
8
+ 2025-04-16 15:26:26,211:amlt-code-runner:INFO - MASTER_ADDR = node-0
9
+ 2025-04-16 15:26:26,211:amlt-code-runner:INFO - MASTER_PORT = 9500
10
+ 2025-04-16 15:26:26,212:amlt-code-runner:WARNING - Installing amlt runtime dependencies: ['wrapt', 'azure-identity', 'python-dateutil', 'pytz'] into /tmp/amlt-user-base
11
+ 2025-04-16 15:26:27,771:amlt-code-runner:INFO - Executing ./amlt_setup.sh, ./amlt_run.sh
12
+ 2025-04-16 15:26:27,838:background_dirsync:INFO - Starting directory syncer from '/scratch/amlt_code/outputs' to '/mnt/output/projects/amlt_project/amlt-results/7255445584.46229-4376bad2-6abd-462d-a8b5-b76415d70254', every 30.000000s
13
+ 2025-04-16 15:26:27,842:background_dirsync:INFO - Starting directory syncer from '/scratch/azureml/cr/j/069eb34b89834120b195c77689a8ea19/exe/wd/logs' to '/scratch/amlt_code/outputs/logs', every 30.000000s
model-00001-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:622a3427f6439cd21589ed3459dffbf55e27d33e8ea77d74fec1b1c7f4abe3ab
3
  size 4943162336
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ed6be2d982a4be8993bb6e48f4b751f320ccaab466acd1f131c6ec2bf5d4b801
3
  size 4943162336
model-00002-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:46544a295ef4e6f971a5b0f5b1c45b19af8d305d43fe6fde7921d80f836b9720
3
  size 4999819336
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e5a1d2e4419867725687b932297921767a520eaf40cedceaed33ed319cf1c48
3
  size 4999819336
model-00003-of-00003.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d79a762c1a70dca518a4b84fa15d987bf58ef39db60d5097f38f1e09216251bb
3
  size 4540516344
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:102413ad10f700d47e64fcd429f690b54f7eb08963291a42bd93e0ab3ee9ef7f
3
  size 4540516344
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:fdd85cc022702bd383516e8d5095c9a541580902476786d7780c27b3a560a6e6
3
  size 7736
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:352d1581d48d124911d09266aed78734035700c250d9f9adeff8a5c2c11e1489
3
  size 7736