I found the problem. I assumed all of the scores were being processed the same way but one of the values in my list did indeed contain an inplace operation due to a bug.
There was no problem with torch.cat
Oddly the error didn’t come up before, but it helped me find a bug so that’s great.