Thanks peterjc123, indeed this was not required. The version of utils and engine scripts were not 100% compatible with CPU. I had to make some small fixes, but now it works.
Unfortunately the problems did not stop. I get a silent error that restarts the kernel in the following bold line. By running the script in Powershell, also no error, just stops execution.
res = {target[“image_id”].item(): output for target, output in zip(targets, outputs)}
evaluator_time = time.time()
coco_evaluator.update(res)
UPDATE: Actually the following line is giving silent error when run for the 2nd time in a loop, so I guess it is a memory problem.
coco_dt = loadRes(self.coco_gt, results) if results else COCO()