2024-04-03 09:50:25,336 INFO StreamThr :117 [internal.py:wandb_internal():86] W&B internal server running at pid: 117, started at: 2024-04-03 09:50:25.335796 2024-04-03 09:50:25,338 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status 2024-04-03 09:50:25,657 INFO WriterThread:117 [datastore.py:open_for_write():87] open: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/run-auh8f8mw.wandb 2024-04-03 09:50:25,658 DEBUG SenderThread:117 [sender.py:send():378] send: header 2024-04-03 09:50:25,662 DEBUG SenderThread:117 [sender.py:send():378] send: run 2024-04-03 09:50:25,924 INFO SenderThread:117 [dir_watcher.py:__init__():211] watching files in: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files 2024-04-03 09:50:25,924 INFO SenderThread:117 [sender.py:_start_run_threads():1099] run started: auh8f8mw with start time 1712137825.336735 2024-04-03 09:50:25,927 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: check_version 2024-04-03 09:50:25,928 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: check_version 2024-04-03 09:50:26,028 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: run_start 2024-04-03 09:50:26,041 DEBUG HandlerThread:117 [system_info.py:__init__():26] System info init 2024-04-03 09:50:26,041 DEBUG HandlerThread:117 [system_info.py:__init__():41] System info init done 2024-04-03 09:50:26,041 INFO HandlerThread:117 [system_monitor.py:start():194] Starting system monitor 2024-04-03 09:50:26,041 INFO SystemMonitor:117 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-04-03 09:50:26,041 INFO HandlerThread:117 [system_monitor.py:probe():214] Collecting system info 2024-04-03 09:50:26,042 INFO SystemMonitor:117 [interfaces.py:start():190] Started cpu monitoring 2024-04-03 09:50:26,043 INFO SystemMonitor:117 [interfaces.py:start():190] Started disk monitoring 2024-04-03 09:50:26,044 INFO SystemMonitor:117 [interfaces.py:start():190] Started gpu monitoring 2024-04-03 09:50:26,044 INFO SystemMonitor:117 [interfaces.py:start():190] Started memory monitoring 2024-04-03 09:50:26,046 INFO SystemMonitor:117 [interfaces.py:start():190] Started network monitoring 2024-04-03 09:50:26,063 DEBUG HandlerThread:117 [system_info.py:probe():150] Probing system 2024-04-03 09:50:26,065 DEBUG HandlerThread:117 [gitlib.py:_init_repo():56] git repository is invalid 2024-04-03 09:50:26,065 DEBUG HandlerThread:117 [system_info.py:probe():198] Probing system done 2024-04-03 09:50:26,065 DEBUG HandlerThread:117 [system_monitor.py:probe():223] {'os': 'Linux-5.15.133+-x86_64-with-glibc2.31', 'python': '3.10.13', 'heartbeatAt': '2024-04-03T09:50:26.063417', 'startedAt': '2024-04-03T09:50:25.328797', 'docker': None, 'cuda': None, 'args': (), 'state': 'running', 'program': 'kaggle.ipynb', 'codePathLocal': None, 'root': '/kaggle/working', 'host': '012a0687e8c4', 'username': 'root', 'executable': '/opt/conda/bin/python3.10', 'cpu_count': 2, 'cpu_count_logical': 4, 'cpu_freq': {'current': 2000.12, 'min': 0.0, 'max': 0.0}, 'cpu_freq_per_core': [{'current': 2000.12, 'min': 0.0, 'max': 0.0}, {'current': 2000.12, 'min': 0.0, 'max': 0.0}, {'current': 2000.12, 'min': 0.0, 'max': 0.0}, {'current': 2000.12, 'min': 0.0, 'max': 0.0}], 'disk': {'/': {'total': 8062.387607574463, 'used': 5533.654918670654}}, 'gpu': 'Tesla T4', 'gpu_count': 2, 'gpu_devices': [{'name': 'Tesla T4', 'memory_total': 16106127360}, {'name': 'Tesla T4', 'memory_total': 16106127360}], 'memory': {'total': 31.35755157470703}} 2024-04-03 09:50:26,065 INFO HandlerThread:117 [system_monitor.py:probe():224] Finished collecting system info 2024-04-03 09:50:26,065 INFO HandlerThread:117 [system_monitor.py:probe():227] Publishing system info 2024-04-03 09:50:26,065 DEBUG HandlerThread:117 [system_info.py:_save_conda():207] Saving list of conda packages installed into the current environment 2024-04-03 09:50:26,926 INFO Thread-12 :117 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/conda-environment.yaml 2024-04-03 09:51:20,941 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/conda-environment.yaml 2024-04-03 09:51:21,451 DEBUG HandlerThread:117 [system_info.py:_save_conda():219] Saving conda packages done 2024-04-03 09:51:21,452 INFO HandlerThread:117 [system_monitor.py:probe():229] Finished publishing system info 2024-04-03 09:51:21,463 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,463 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:21,463 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,463 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:21,463 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,464 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:21,464 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,464 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:21,464 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,464 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:21,464 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,464 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: python_packages 2024-04-03 09:51:21,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: internal_messages 2024-04-03 09:51:21,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: stop_status 2024-04-03 09:51:21,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:21,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:21,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:21,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,466 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:21,466 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:21,466 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: python_packages 2024-04-03 09:51:21,469 DEBUG SenderThread:117 [sender.py:send():378] send: telemetry 2024-04-03 09:51:21,470 DEBUG SenderThread:117 [sender.py:send():378] send: config 2024-04-03 09:51:21,471 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:21,472 DEBUG SenderThread:117 [sender.py:send():378] send: telemetry 2024-04-03 09:51:21,472 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:21,472 WARNING SenderThread:117 [sender.py:send_metric():1316] Seen metric with glob (shouldn't happen) 2024-04-03 09:51:21,472 DEBUG SenderThread:117 [sender.py:send():378] send: telemetry 2024-04-03 09:51:21,472 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: stop_status 2024-04-03 09:51:21,543 DEBUG SenderThread:117 [sender.py:send():378] send: files 2024-04-03 09:51:21,543 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-metadata.json with policy now 2024-04-03 09:51:21,941 INFO Thread-12 :117 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-metadata.json 2024-04-03 09:51:21,942 INFO Thread-12 :117 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:51:21,942 INFO Thread-12 :117 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/requirements.txt 2024-04-03 09:51:21,962 INFO wandb-upload_0:117 [upload_job.py:push():131] Uploaded file /tmp/tmpaowk4qp3wandb/lfomoext-wandb-metadata.json 2024-04-03 09:51:22,059 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:23,942 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:51:26,046 DEBUG SystemMonitor:117 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-04-03 09:51:26,047 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:51:26,053 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:26,944 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/config.yaml 2024-04-03 09:51:27,060 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:31,190 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:32,061 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:36,191 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:37,062 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:41,192 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:42,063 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:46,193 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:47,063 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:47,347 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:51:47,349 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:47,350 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:47,350 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:47,350 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:47,350 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:51:47,350 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:51:47,352 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:51:47,951 INFO Thread-12 :117 [dir_watcher.py:_on_file_created():271] file/dir created: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:51:51,353 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:52,064 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:52,111 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:51:52,113 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:52,113 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:52,113 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:52,113 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 09:51:52,113 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:51:52,114 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:51:52,114 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:51:52,953 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:51:53,954 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:51:56,048 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:51:57,054 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:51:57,065 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:51:57,955 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/config.yaml 2024-04-03 09:52:02,066 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:02,204 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:07,067 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:07,205 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:12,068 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:12,205 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:17,069 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:17,206 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:22,070 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:22,207 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:26,049 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:52:27,071 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:28,050 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:32,072 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:33,051 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:37,073 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:38,052 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:42,074 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:42,266 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:52:42,267 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:52:42,267 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:52:42,269 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:52:42,972 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:52:43,270 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:47,075 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:47,098 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:52:47,099 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:52:47,099 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:52:47,099 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:52:47,974 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:52:48,279 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:49,975 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:52:52,076 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:53,280 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:52:56,050 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:52:57,077 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:52:59,051 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:02,078 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:04,052 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:07,079 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:09,053 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:12,081 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:14,054 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:17,082 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:19,055 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:22,083 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:24,056 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:26,051 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:53:27,084 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:30,052 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:32,085 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:35,053 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:37,086 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:37,474 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:53:37,475 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:53:37,475 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:53:37,477 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:53:37,992 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:53:40,478 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:42,087 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:42,295 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:53:42,297 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:53:42,297 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:53:42,298 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:53:42,994 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:53:43,994 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:53:45,482 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:47,088 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:50,483 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:52,089 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:53:55,483 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:53:56,052 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:53:57,090 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:01,053 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:02,091 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:06,054 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:07,092 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:11,055 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:12,093 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:16,056 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:17,094 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:21,056 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:22,095 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:26,053 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:54:27,054 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:27,096 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:32,055 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:32,097 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:33,533 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:54:33,534 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:54:33,535 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:54:33,535 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:54:34,013 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:54:37,098 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:37,536 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:38,354 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:54:38,355 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:54:38,356 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:54:38,358 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:54:39,015 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:54:40,015 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:54:42,099 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:42,568 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:47,100 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:47,569 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:52,101 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:52,570 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:54:56,054 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:54:57,102 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:54:58,055 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:02,103 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:03,056 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:07,104 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:08,057 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:12,105 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:13,058 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:17,106 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:18,058 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:22,106 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:23,059 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:26,055 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:55:27,107 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:29,056 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:29,914 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:55:29,915 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:55:29,916 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:55:29,916 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:55:30,034 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:55:32,108 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:34,740 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:55:34,741 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:55:34,741 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:55:34,742 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:34,742 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:55:35,035 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:55:36,036 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:55:37,109 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:40,016 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:42,110 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:45,017 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:47,111 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:50,018 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:52,112 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:55:55,019 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:55:56,055 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:55:57,113 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:00,057 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:02,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:05,057 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:07,115 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:10,058 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:12,116 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:15,059 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:17,117 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:20,060 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:22,117 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:25,061 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:25,854 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:56:25,855 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:56:25,856 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:56:25,856 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:56:26,054 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:56:26,056 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:56:27,118 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:30,671 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:56:30,672 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:56:30,673 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:56:30,673 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:30,674 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:56:31,056 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:56:32,057 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:56:32,119 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:35,860 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:37,120 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:40,861 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:42,123 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:45,861 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:47,124 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:50,862 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:52,125 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:56:55,863 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:56:56,057 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:56:57,126 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:01,059 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:02,126 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:06,059 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:07,127 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:11,060 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:12,128 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:16,061 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:17,129 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:21,062 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:21,226 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:57:21,228 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:57:21,228 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:57:21,228 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:57:22,074 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:57:22,130 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:26,038 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:57:26,039 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:57:26,039 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:57:26,040 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:57:26,058 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:57:26,075 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:57:26,234 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:27,131 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:28,076 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:57:31,235 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:32,132 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:36,236 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:37,133 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:41,236 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:42,134 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:46,237 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:47,135 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:51,238 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:52,136 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:57:56,059 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:57:57,060 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:57:57,137 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:02,061 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:02,138 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:07,062 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:07,139 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:12,063 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:12,140 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:17,064 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:17,141 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:17,231 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:58:17,232 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:58:17,232 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:58:17,232 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:58:18,094 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:58:22,033 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:58:22,034 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:58:22,035 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:58:22,035 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:58:22,095 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:58:22,142 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:22,229 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:24,096 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:58:26,060 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:58:27,143 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:28,061 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:32,145 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:33,062 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:37,146 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:38,063 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:42,147 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:43,063 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:47,148 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:48,065 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:52,149 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:53,066 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:58:56,061 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:58:57,150 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:58:59,062 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:02,151 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:04,063 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:07,152 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:09,064 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:12,033 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:59:12,034 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:59:12,034 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:59:12,035 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:59:12,113 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:59:12,153 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:15,036 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:16,847 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 09:59:16,848 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 09:59:16,849 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 09:59:16,849 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 09:59:17,115 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 09:59:17,154 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:18,116 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 09:59:20,039 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:22,155 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:25,040 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:26,062 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:59:27,156 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:30,063 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:32,157 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:35,064 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:37,158 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:40,065 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:42,159 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:45,066 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:47,160 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:50,067 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:52,161 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 09:59:55,069 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 09:59:56,067 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 09:59:57,162 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:01,068 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:02,167 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:06,069 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:07,168 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:07,817 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:00:07,818 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:00:07,818 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:00:07,819 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:00:08,133 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:00:11,820 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:12,169 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:12,618 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:00:12,619 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:00:12,620 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:00:12,620 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:00:13,135 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:00:14,136 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:00:17,170 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:17,816 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:22,171 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:22,817 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:26,064 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:00:27,173 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:28,065 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:32,174 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:33,066 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:37,175 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:38,067 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:42,176 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:43,068 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:47,177 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:48,069 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:52,177 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:53,070 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:00:56,067 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:00:57,178 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:00:59,068 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:02,179 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:03,030 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:01:03,031 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:01:03,032 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:01:03,032 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:01:03,153 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:01:05,033 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:07,180 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:07,842 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:01:07,843 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:01:07,844 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:01:07,844 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:01:08,155 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:01:11,030 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:11,157 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:01:12,181 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:16,031 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:17,182 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:21,032 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:22,183 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:26,032 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:26,066 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:01:27,184 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:31,067 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:32,185 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:36,068 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:37,186 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:41,069 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:42,187 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:46,070 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:47,188 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:51,071 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:52,189 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:56,067 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:01:57,068 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:01:57,189 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:01:58,117 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:01:58,118 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:01:58,119 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:01:58,119 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:01:58,176 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:02:02,120 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:02,190 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:02,923 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:02:02,924 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:02:02,925 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:02:02,925 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:02:03,177 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:02:05,178 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:02:07,171 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:07,191 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:12,169 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:12,192 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:17,170 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:17,193 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:22,171 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:22,194 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:26,069 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:02:27,195 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:28,070 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:32,196 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:33,070 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:37,197 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:38,071 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:42,198 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:43,073 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:47,199 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:48,073 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:52,199 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:53,074 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:53,115 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:02:53,116 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:02:53,116 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:02:53,117 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:02:53,195 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:02:56,069 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:02:57,200 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:02:57,921 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:02:57,922 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:02:57,922 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:02:57,923 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:02:58,110 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:02:58,197 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:03:01,198 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:03:02,201 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:03,111 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:07,202 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:08,111 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:12,203 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:13,112 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:17,204 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:18,113 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:22,205 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:23,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:26,070 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:03:27,206 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:29,071 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:32,207 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:34,072 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:37,211 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:39,073 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:42,212 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:44,074 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:47,214 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:48,915 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:03:48,916 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:03:48,916 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:03:48,916 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:03:49,215 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:03:49,917 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:52,214 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:53,728 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:03:53,730 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:03:53,730 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:03:53,730 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:03:54,216 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:03:54,921 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:03:56,071 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:03:57,215 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:03:57,218 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:04:00,072 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:02,216 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:05,073 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:07,217 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:10,074 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:12,218 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:15,075 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:17,219 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:20,076 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:22,220 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:25,077 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:26,072 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:04:27,221 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:31,073 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:32,222 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:36,074 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:37,223 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:41,075 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:42,224 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:44,199 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:04:44,200 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:04:44,200 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:04:44,201 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:04:44,234 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:04:46,201 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:47,224 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:49,013 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:04:49,014 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:04:49,014 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:04:49,015 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:04:49,236 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:04:51,237 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:04:51,319 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:52,225 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:04:56,073 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:04:57,074 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:04:57,226 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:02,075 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:02,227 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:07,076 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:07,228 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:12,077 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:12,229 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:17,078 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:17,230 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:22,079 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:22,231 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:26,074 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:05:27,232 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:28,075 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:32,233 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:33,076 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:37,234 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:38,077 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:40,060 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:05:40,061 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:05:40,062 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:05:40,062 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:05:40,255 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:05:42,235 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:44,063 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:44,873 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:05:44,874 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:05:44,874 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:05:44,874 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:05:45,257 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:05:47,236 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:47,257 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:05:49,146 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:52,237 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:05:54,147 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:05:56,075 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:05:57,238 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:00,076 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:02,239 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:05,077 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:07,240 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:10,078 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:12,241 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:15,079 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:17,242 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:20,080 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:22,243 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:25,081 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:26,076 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:06:27,244 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:31,077 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:32,245 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:34,804 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:06:34,805 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:06:34,805 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:06:34,806 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:06:35,273 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:06:36,807 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:37,246 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:39,604 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:06:39,605 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:06:39,605 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:06:39,606 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:06:40,275 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:06:41,275 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:06:41,940 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:42,247 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:46,942 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:47,248 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:51,943 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:52,249 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:06:56,077 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:06:57,078 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:06:57,250 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:02,079 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:02,251 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:07,080 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:07,252 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:12,081 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:12,253 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:17,081 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:17,254 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:22,082 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:22,255 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:26,078 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:07:27,256 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:28,079 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:29,640 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:07:29,641 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:07:29,641 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:07:29,642 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:07:30,293 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:07:32,257 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:33,643 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:34,449 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:07:34,450 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:07:34,450 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:07:34,451 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:07:35,295 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:07:37,258 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:37,327 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:07:38,700 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:42,259 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:43,701 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:47,260 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:48,701 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:52,261 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:53,702 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:07:56,079 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:07:57,262 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:07:59,080 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:02,263 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:04,081 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:07,264 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:09,082 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:12,265 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:14,083 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:17,265 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:19,084 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:22,266 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:24,084 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:26,080 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:08:26,301 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:08:26,302 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:08:26,302 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:08:26,303 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:08:26,344 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:08:27,267 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:29,304 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:31,109 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:08:31,110 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:08:31,111 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:08:31,111 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:08:31,346 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:08:32,273 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:33,346 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:08:34,324 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:37,274 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:39,325 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:42,275 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:44,326 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:47,276 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:49,326 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:52,277 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:08:54,327 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:08:56,081 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:08:57,278 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:00,082 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:02,279 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:05,083 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:07,280 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:10,084 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:12,281 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:15,085 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:17,282 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:20,086 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:21,254 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:09:21,256 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:09:21,256 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:09:21,256 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:09:21,368 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:09:22,283 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:25,257 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:26,063 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:09:26,065 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:09:26,065 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:09:26,066 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:09:26,082 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:09:26,370 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:09:27,284 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:27,370 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:09:30,268 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:32,285 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:35,268 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:37,286 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:40,269 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:42,287 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:45,270 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:47,287 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:50,271 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:52,288 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:09:55,272 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:09:56,083 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:09:57,289 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:01,084 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:02,290 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:06,085 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:07,291 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:11,086 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:12,292 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:16,087 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:17,293 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:17,595 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:10:17,596 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:10:17,596 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:10:17,596 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:10:18,388 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:10:21,597 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:22,294 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:22,404 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:10:22,405 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:10:22,406 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:10:22,406 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:10:23,390 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:10:25,391 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:10:26,084 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:10:27,084 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:27,295 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:32,085 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:32,296 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:37,086 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:37,297 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:42,087 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:42,298 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:47,088 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:47,299 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:52,089 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:10:52,300 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:56,084 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:10:57,301 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:10:58,086 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:02,301 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:03,086 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:07,302 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:08,087 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:12,303 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:13,088 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:14,046 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:11:14,047 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:11:14,047 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:11:14,048 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:11:14,407 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:11:17,304 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:18,851 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:11:18,852 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:11:18,853 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:11:18,853 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:18,853 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:11:19,409 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:11:21,410 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:11:22,305 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:24,061 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:26,085 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:11:27,306 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:29,086 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:32,307 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:34,087 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:37,308 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:39,088 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:42,309 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:44,089 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:47,310 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:49,090 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:52,310 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:11:54,091 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:11:56,086 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:11:57,311 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:00,088 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:02,312 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:05,089 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:07,313 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:09,420 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:12:09,421 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:12:09,421 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:12:09,421 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:12:09,426 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:12:10,422 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:12,314 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:14,233 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:12:14,234 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:12:14,234 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:12:14,234 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:12:14,428 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:12:15,428 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:15,429 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:12:17,315 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:20,429 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:22,316 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:25,429 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:26,088 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:12:27,316 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:31,089 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:32,317 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:36,090 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:37,318 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:41,091 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:42,319 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:46,091 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:47,320 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:51,092 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:52,321 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:12:56,089 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:12:57,089 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:12:57,322 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:02,090 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:02,323 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:05,402 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:13:05,403 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:13:05,404 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:13:05,404 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:13:05,445 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:13:07,324 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:07,405 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:10,210 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:13:10,212 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:13:10,212 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:13:10,212 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:13:10,448 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:13:11,448 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:13:12,325 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:12,407 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:17,326 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:17,408 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:22,327 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:22,408 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:26,089 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:13:27,328 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:28,090 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:32,330 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:33,091 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:37,331 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:38,092 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:42,333 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:43,093 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:47,333 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:48,094 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:52,334 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:53,095 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:13:56,090 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:13:57,335 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:13:59,091 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:00,963 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:14:00,965 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:14:00,965 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:14:00,966 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:14:01,466 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:14:02,336 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:04,967 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:05,767 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:14:05,768 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:14:05,768 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:14:05,769 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:14:06,468 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:14:07,337 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:07,468 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:14:10,013 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:12,338 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:15,014 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:17,339 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:20,015 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:22,340 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:25,016 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:26,091 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:14:27,341 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:30,092 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:32,342 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:35,093 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:37,343 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:40,094 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:42,343 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:45,095 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:47,344 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:50,096 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:52,345 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:14:55,096 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:14:56,001 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:14:56,002 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:14:56,002 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:14:56,003 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:14:56,092 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:14:56,485 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:14:57,346 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:00,796 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:15:00,797 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:15:00,798 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:15:00,798 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:00,799 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:15:01,487 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:15:02,347 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:03,488 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:15:05,996 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:07,348 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:10,997 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:12,349 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:15,998 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:17,350 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:20,999 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:22,351 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:25,999 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:26,093 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:15:27,352 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:31,095 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:32,353 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:36,095 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:37,354 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:41,096 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:42,355 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:46,097 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:47,356 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:51,098 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:51,847 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:15:51,848 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:15:51,849 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:15:51,849 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:15:52,357 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:52,507 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:15:56,094 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:15:56,651 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:15:56,652 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:15:56,653 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:15:56,653 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:15:56,654 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:15:57,358 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:15:57,509 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:15:59,510 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:16:02,039 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:02,359 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:07,040 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:07,359 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:12,041 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:12,360 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:17,042 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:17,361 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:22,043 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:22,362 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:26,095 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:16:27,096 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:27,363 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:32,097 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:32,364 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:37,098 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:37,365 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:42,098 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:42,366 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:47,099 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:47,367 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:47,889 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:16:47,891 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:16:47,891 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:16:47,891 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:16:48,529 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:16:52,368 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:52,688 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:16:52,689 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:16:52,689 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:16:52,690 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:16:52,690 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:16:53,531 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:16:55,532 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:16:56,096 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:16:57,369 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:16:58,097 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:02,370 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:03,098 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:07,371 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:08,099 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:12,372 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:13,100 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:17,373 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:18,101 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:22,374 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:23,102 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:26,097 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:17:27,375 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:29,098 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:32,375 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:34,099 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:37,377 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:39,100 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:42,378 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:44,101 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:44,194 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:17:44,196 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:17:44,196 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:17:44,196 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:17:44,551 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:17:47,379 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:48,997 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:17:48,999 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:17:48,999 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:17:49,000 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:17:49,225 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:49,553 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:17:51,554 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:17:52,380 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:17:54,227 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:17:56,098 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:17:57,381 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:00,099 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:02,382 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:05,100 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:07,383 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:10,101 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:12,384 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:15,102 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:17,385 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:20,103 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:22,386 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:25,103 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:26,099 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:18:27,387 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:31,100 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:32,388 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:36,101 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:37,389 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:39,817 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:18:39,818 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:18:39,818 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:18:39,819 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:18:40,573 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:18:41,820 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:42,391 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:44,620 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:18:44,621 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:18:44,621 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:18:44,622 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:18:45,575 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:18:45,576 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:18:46,908 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:47,392 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:51,909 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:52,393 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:18:56,099 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:18:57,100 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:18:57,393 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:02,101 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:02,394 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:07,102 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:07,395 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:12,103 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:12,396 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:17,104 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:17,397 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:22,104 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:22,398 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:26,100 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:19:27,399 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:28,101 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:32,400 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:33,102 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:35,345 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:19:35,346 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:19:35,347 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:19:35,347 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:19:35,594 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:19:37,401 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:38,348 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:40,162 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:19:40,163 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:19:40,164 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:19:40,164 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:19:40,595 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:19:41,596 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:19:42,407 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:43,439 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:47,408 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:48,440 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:52,409 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:53,441 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:19:56,102 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:19:57,409 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:19:59,103 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:02,410 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:04,104 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:07,411 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:09,105 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:12,412 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:14,106 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:17,413 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:19,106 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:22,414 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:24,108 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:26,103 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:20:27,415 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:30,104 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:31,203 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:20:31,204 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:20:31,204 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:20:31,205 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:20:31,613 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:20:32,416 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:35,206 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:36,012 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:20:36,014 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:20:36,014 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:20:36,014 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:20:36,615 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:20:37,417 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:37,615 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:20:40,224 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:42,417 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:45,225 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:47,418 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:50,225 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:52,419 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:20:55,226 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:20:56,103 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:20:57,420 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:01,105 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:02,421 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:06,105 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:07,422 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:11,107 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:12,423 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:16,108 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:17,424 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:21,109 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:22,425 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:26,104 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:21:26,315 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:21:26,317 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:21:26,318 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:21:26,318 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:26,319 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:21:26,633 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:21:27,426 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:31,127 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:21:31,128 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:21:31,128 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:21:31,129 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:21:31,393 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:31,634 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:21:32,427 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:33,635 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:21:36,394 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:37,427 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:41,395 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:42,428 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:46,396 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:47,429 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:51,397 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:52,430 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:21:56,105 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:21:57,106 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:21:57,431 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:02,107 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:02,432 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:07,108 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:07,433 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:12,109 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:12,434 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:17,109 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:17,435 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:21,896 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:22:21,897 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:22:21,898 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:22:21,898 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:22:22,436 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:22,652 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:22:22,899 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:26,106 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:22:26,689 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:22:26,690 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:22:26,691 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:22:26,691 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:22:27,437 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:27,654 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:22:27,654 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:22:27,933 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:32,438 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:32,934 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:37,439 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:37,934 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:42,440 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:42,935 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:47,441 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:47,936 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:52,442 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:52,937 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:22:56,111 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:22:57,443 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:22:58,112 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:02,444 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:03,113 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:07,445 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:08,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:12,446 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:13,115 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:17,447 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:18,055 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:23:18,057 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:23:18,057 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:23:18,057 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:23:18,670 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:23:19,058 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:22,448 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:22,859 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:23:22,860 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:23:22,860 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:23:22,861 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:23:23,672 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:23:23,673 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:23:25,048 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:26,108 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:23:27,449 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:30,110 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:32,450 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:35,110 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:37,451 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:40,111 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:42,452 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:45,112 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:47,453 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:50,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:52,454 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:23:55,115 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:23:56,110 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:23:57,455 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:01,112 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:02,456 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:06,113 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:07,457 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:11,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:12,458 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:14,150 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:24:14,151 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:24:14,151 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:24:14,152 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:24:14,691 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:24:16,153 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:17,459 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:18,955 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:24:18,956 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:24:18,957 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:24:18,957 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:24:19,693 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:24:19,693 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:24:21,181 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:22,460 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:26,110 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:24:27,111 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:27,461 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:32,112 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:32,462 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:37,113 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:37,463 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:42,113 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:42,464 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:47,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:47,465 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:52,115 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:24:52,466 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:56,111 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:24:57,467 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:24:58,112 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:02,468 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:03,113 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:07,469 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:08,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:09,524 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:25:09,526 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:25:09,526 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:25:09,526 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:25:09,711 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:25:12,470 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:13,527 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:14,321 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:25:14,322 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:25:14,323 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:25:14,323 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:25:14,713 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:25:15,714 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:25:17,476 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:19,526 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:22,477 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:24,527 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:26,111 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:25:27,482 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:30,113 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:32,483 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:35,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:37,484 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:40,115 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:42,485 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:45,116 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:47,486 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:50,117 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:52,487 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:25:55,118 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:25:56,112 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:25:57,488 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:01,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:02,489 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:05,330 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:26:05,331 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:26:05,332 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:26:05,332 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:26:05,732 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:26:06,333 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:07,490 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:10,124 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:26:10,125 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:26:10,125 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:26:10,126 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:26:10,734 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:26:11,415 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:12,491 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:12,735 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:26:16,416 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:17,492 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:21,417 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:22,493 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:26,113 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:26:27,114 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:27,494 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:32,115 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:32,495 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:37,116 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:37,495 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:42,117 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:42,496 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:47,117 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:47,497 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:52,118 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:26:52,498 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:56,114 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:26:57,499 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:26:58,115 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:00,761 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:27:00,763 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:27:00,763 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:27:00,763 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:27:01,753 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:27:02,500 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:03,764 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:05,559 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:27:05,561 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:27:05,561 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:27:05,562 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:27:05,755 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:27:07,501 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:08,756 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:27:08,767 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:12,502 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:13,767 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:17,503 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:18,768 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:22,504 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:23,769 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:26,115 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:27:27,505 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:29,117 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:32,505 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:34,118 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:37,506 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:39,118 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:42,507 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:44,119 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:47,508 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:49,120 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:52,509 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:27:54,121 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:27:56,023 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:27:56,024 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:27:56,025 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:27:56,025 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:27:56,116 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:27:56,775 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:27:57,510 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:00,117 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:00,834 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:28:00,835 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:28:00,835 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:28:00,836 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:28:01,777 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:28:02,511 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:02,777 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:28:05,140 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:07,513 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:10,141 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:12,514 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:15,142 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:17,514 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:20,143 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:22,515 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:25,144 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:26,118 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:28:27,516 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:31,119 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:32,517 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:36,120 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:37,518 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:41,121 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:42,519 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:46,122 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:47,520 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:51,040 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:28:51,041 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:28:51,041 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:28:51,042 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:28:51,795 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:28:52,042 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:52,521 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:55,837 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:28:55,838 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:28:55,839 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:28:55,839 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:28:56,118 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:28:56,798 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:28:57,119 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:28:57,522 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:28:58,798 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:29:02,120 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:02,523 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:07,121 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:07,524 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:12,127 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:12,525 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:17,128 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:17,526 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:22,129 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:22,527 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:26,119 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:29:27,528 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:28,120 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:32,529 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:33,121 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:37,530 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:38,122 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:42,531 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:43,123 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:47,144 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:29:47,145 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:29:47,145 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:29:47,146 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:29:47,532 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:47,817 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:29:48,146 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:51,957 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:29:51,958 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:29:51,958 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:29:51,959 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:29:52,534 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:52,819 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:29:53,159 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:29:54,820 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:29:56,120 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:29:57,535 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:29:59,121 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:02,536 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:04,122 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:07,536 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:09,123 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:12,537 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:14,124 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:17,538 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:19,125 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:22,539 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:24,126 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:26,121 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:30:27,540 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:30,122 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:32,541 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:35,123 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:37,542 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:40,124 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:42,543 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:43,208 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:30:43,209 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:30:43,209 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:30:43,210 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:30:43,837 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:30:45,210 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:47,544 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:48,023 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:30:48,024 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:30:48,025 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:30:48,026 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:30:48,839 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:30:50,244 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:50,840 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:30:52,545 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:30:55,244 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:30:56,122 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:30:57,546 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:01,123 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:02,547 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:06,124 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:07,548 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:11,125 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:12,549 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:16,126 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:17,550 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:21,127 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:22,551 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:26,123 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:31:27,124 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:27,552 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:32,124 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:32,553 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:37,125 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:37,554 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:38,731 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:31:38,732 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:31:38,733 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:31:38,733 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:31:38,857 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:31:42,555 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:42,734 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:43,535 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:31:43,536 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:31:43,537 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:31:43,537 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:31:43,858 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:31:44,859 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:31:47,561 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:48,730 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:52,562 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:53,731 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:31:56,124 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:31:57,563 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:31:59,125 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:02,564 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:04,126 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:07,565 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:09,127 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:12,566 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:14,128 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:17,567 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:19,129 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:22,568 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:24,130 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:26,125 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:32:27,569 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:30,126 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:32,570 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:33,914 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:32:33,916 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:32:33,916 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:32:33,917 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:32:34,877 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:32:35,917 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:37,571 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:38,716 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:32:38,717 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:32:38,717 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:32:38,718 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:32:38,879 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:32:40,879 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:32:40,919 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:42,572 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:45,920 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:47,573 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:50,921 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:52,574 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:32:55,922 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:32:56,125 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:32:57,575 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:01,127 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:02,576 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:06,127 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:07,577 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:11,128 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:12,578 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:16,130 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:17,579 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:21,130 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:22,580 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:26,126 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:33:27,127 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:27,581 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:29,207 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:33:29,208 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:33:29,208 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:33:29,209 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:33:29,898 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:33:32,210 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:32,582 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:34,009 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:33:34,010 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:33:34,010 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:33:34,010 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:33:34,899 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:33:36,900 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:33:37,583 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:38,210 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:42,584 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:43,211 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:47,584 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:48,212 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:52,585 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:53,212 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:33:56,127 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:33:57,586 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:33:59,129 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:02,587 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:04,130 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:07,588 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:09,131 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:12,589 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:14,131 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:17,590 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:19,132 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:22,591 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:24,133 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:24,852 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:34:24,853 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:34:24,853 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:34:24,853 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:34:24,917 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:34:26,133 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:34:27,592 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:29,134 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:29,650 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:34:29,651 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:34:29,652 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:34:29,652 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:34:29,919 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:34:30,920 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/output.log 2024-04-03 10:34:32,593 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:34,846 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:37,594 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:39,847 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:42,595 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:44,848 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:47,596 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:49,849 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:52,597 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:54,850 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:34:56,130 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:34:57,598 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:34:59,223 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:34:59,224 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:34:59,224 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:34:59,226 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:34:59,930 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:35:00,227 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:02,601 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:04,020 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:35:04,022 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:35:04,022 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:35:04,022 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:35:04,233 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: partial_history 2024-04-03 10:35:04,235 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 10:35:04,235 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 10:35:04,235 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 10:35:04,235 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 10:35:04,236 DEBUG SenderThread:117 [sender.py:send():378] send: metric 2024-04-03 10:35:04,236 DEBUG SenderThread:117 [sender.py:send():378] send: history 2024-04-03 10:35:04,236 DEBUG SenderThread:117 [sender.py:send_request():405] send_request: summary_record 2024-04-03 10:35:04,236 INFO SenderThread:117 [sender.py:_save_file():1365] saving file wandb-summary.json with policy end 2024-04-03 10:35:04,237 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: pause 2024-04-03 10:35:04,238 INFO HandlerThread:117 [handler.py:handle_request_pause():708] stopping system metrics thread 2024-04-03 10:35:04,238 INFO HandlerThread:117 [system_monitor.py:finish():203] Stopping system monitor 2024-04-03 10:35:04,238 DEBUG SystemMonitor:117 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-04-03 10:35:04,238 INFO HandlerThread:117 [interfaces.py:finish():202] Joined cpu monitor 2024-04-03 10:35:04,238 DEBUG SystemMonitor:117 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-04-03 10:35:04,239 INFO HandlerThread:117 [interfaces.py:finish():202] Joined disk monitor 2024-04-03 10:35:04,249 INFO HandlerThread:117 [interfaces.py:finish():202] Joined gpu monitor 2024-04-03 10:35:04,249 INFO HandlerThread:117 [interfaces.py:finish():202] Joined memory monitor 2024-04-03 10:35:04,249 INFO HandlerThread:117 [interfaces.py:finish():202] Joined network monitor 2024-04-03 10:35:04,250 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:35:04,932 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/wandb-summary.json 2024-04-03 10:35:05,250 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:07,602 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:10,251 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:12,603 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:15,252 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:17,604 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:20,258 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:21,938 INFO Thread-12 :117 [dir_watcher.py:_on_file_modified():288] file/dir modified: /kaggle/working/wandb/run-20240403_095025-auh8f8mw/files/config.yaml 2024-04-03 10:35:22,605 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:26,636 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:27,606 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:31,637 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:32,608 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:36,638 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:37,609 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:41,638 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:42,610 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:46,639 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:47,611 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:51,640 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:52,613 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:35:56,640 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:35:57,614 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:01,641 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:02,615 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:06,642 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:07,616 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:11,643 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:12,617 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:16,645 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:17,618 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:21,645 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:22,619 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:26,646 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:27,620 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:31,647 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:32,622 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:36,648 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:37,623 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:41,649 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:42,624 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:46,650 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:47,625 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:51,650 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:52,626 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:36:56,651 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:36:57,627 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:01,652 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:02,628 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:06,653 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:07,629 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:11,654 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:12,630 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:16,655 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:17,632 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:21,656 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:22,633 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:26,657 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:27,634 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:31,657 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:32,635 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:36,658 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:37,636 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:41,659 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:42,637 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:46,660 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:47,638 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:51,661 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:52,639 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:37:56,662 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:37:57,640 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:01,662 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:02,641 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:06,663 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:07,642 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:11,664 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:12,643 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:16,665 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:17,644 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:21,666 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:22,645 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:24,987 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: resume 2024-04-03 10:38:24,987 INFO HandlerThread:117 [handler.py:handle_request_resume():699] starting system metrics thread 2024-04-03 10:38:24,987 INFO HandlerThread:117 [system_monitor.py:start():194] Starting system monitor 2024-04-03 10:38:24,987 INFO SystemMonitor:117 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-04-03 10:38:24,988 INFO SystemMonitor:117 [interfaces.py:start():190] Started cpu monitoring 2024-04-03 10:38:24,988 INFO SystemMonitor:117 [interfaces.py:start():190] Started disk monitoring 2024-04-03 10:38:24,989 INFO SystemMonitor:117 [interfaces.py:start():190] Started gpu monitoring 2024-04-03 10:38:24,989 INFO SystemMonitor:117 [interfaces.py:start():190] Started memory monitoring 2024-04-03 10:38:24,990 INFO SystemMonitor:117 [interfaces.py:start():190] Started network monitoring 2024-04-03 10:38:25,015 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: pause 2024-04-03 10:38:25,015 INFO HandlerThread:117 [handler.py:handle_request_pause():708] stopping system metrics thread 2024-04-03 10:38:25,015 INFO HandlerThread:117 [system_monitor.py:finish():203] Stopping system monitor 2024-04-03 10:38:25,015 DEBUG SystemMonitor:117 [system_monitor.py:_start():172] Starting system metrics aggregation loop 2024-04-03 10:38:25,016 DEBUG SystemMonitor:117 [system_monitor.py:_start():179] Finished system metrics aggregation loop 2024-04-03 10:38:25,016 DEBUG SystemMonitor:117 [system_monitor.py:_start():183] Publishing last batch of metrics 2024-04-03 10:38:25,017 INFO HandlerThread:117 [interfaces.py:finish():202] Joined cpu monitor 2024-04-03 10:38:25,017 INFO HandlerThread:117 [interfaces.py:finish():202] Joined disk monitor 2024-04-03 10:38:25,029 INFO HandlerThread:117 [interfaces.py:finish():202] Joined gpu monitor 2024-04-03 10:38:25,029 INFO HandlerThread:117 [interfaces.py:finish():202] Joined memory monitor 2024-04-03 10:38:25,029 INFO HandlerThread:117 [interfaces.py:finish():202] Joined network monitor 2024-04-03 10:38:25,030 DEBUG SenderThread:117 [sender.py:send():378] send: stats 2024-04-03 10:38:27,030 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:27,646 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:32,031 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:32,647 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:37,032 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:37,648 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:42,033 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:42,649 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:47,034 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:47,650 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:49,977 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: resume 2024-04-03 10:38:49,977 INFO HandlerThread:117 [handler.py:handle_request_resume():699] starting system metrics thread 2024-04-03 10:38:49,977 INFO HandlerThread:117 [system_monitor.py:start():194] Starting system monitor 2024-04-03 10:38:49,978 INFO SystemMonitor:117 [system_monitor.py:_start():158] Starting system asset monitoring threads 2024-04-03 10:38:49,978 INFO SystemMonitor:117 [interfaces.py:start():190] Started cpu monitoring 2024-04-03 10:38:49,979 INFO SystemMonitor:117 [interfaces.py:start():190] Started disk monitoring 2024-04-03 10:38:49,980 INFO SystemMonitor:117 [interfaces.py:start():190] Started gpu monitoring 2024-04-03 10:38:49,981 INFO SystemMonitor:117 [interfaces.py:start():190] Started memory monitoring 2024-04-03 10:38:49,981 INFO SystemMonitor:117 [interfaces.py:start():190] Started network monitoring 2024-04-03 10:38:52,035 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:52,651 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive 2024-04-03 10:38:57,036 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: status_report 2024-04-03 10:38:57,652 DEBUG HandlerThread:117 [handler.py:handle_request():146] handle_request: keepalive