@ptrblck
Training Epoch 0:  71%|████████████████████████████████████████▋                | 82/115 [05:18<01:00,  1.83s/it]/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:84: operator(): block: [0,0,0], thread: [0,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:84: operator(): block: [0,0,0], thread: [2,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:84: operator(): block: [0,0,0], thread: [3,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:84: operator(): block: [0,0,0], thread: [4,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:84: operator(): block: [0,0,0], thread: [26,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
/pytorch/aten/src/ATen/native/cuda/IndexKernel.cu:84: operator(): block: [0,0,0], thread: [28,0,0] Assertion `index >= -sizes[i] && index < sizes[i] && "index out of bounds"` failed.
Training Epoch 0:  71%|████████████████████████████████████████▋                | 82/115 [05:33<02:14,  4.07s/it]
Traceback (most recent call last):
  File "trainer.py", line 235, in <module>
    trainer(opt.data_config, opt.multiscale_training, opt.img_size, opt.batch_size, opt.n_cpu, opt.model_def, opt.pretrained_weights,
  File "trainer.py", line 191, in trainer
    train_result = train(model, optimizer, train_dataloader, epoch, device,  gradient_accumulations)
  File "trainer.py", line 96, in train
    loss, outputs = model(imgs, targets)
  File "~~~/.pyenv/versions/yoloenv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "~~~~~/models.py", line 266, in forward
    x, layer_loss = module[0](x, targets, img_dim)
  File "~~~/.pyenv/versions/yoloenv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File ~~~~/models.py", line 191, in forward
    iou_scores, class_mask, obj_mask, noobj_mask, tx, ty, tw, th, tcls, tconf = build_targets(
  File "~~~~~/utils/utils.py", line 301, in build_targets
    obj_mask[b, best_n, gj, gi] = 1
RuntimeError: CUDA error: device-side assert triggered

This is the result error of CUDA_LAUNCH_BLOCKING=1 python script.py args. I think it’s the right line of code causing this assertion, but I don’t know what error is this.
And surprisingly, this code runs well when I use coco dataset with only 1 class(person).