So I’ve made headway, I think.  Starting with my test loop:

Run the testing batches
with torch.no_grad(): # do not update weights and bias
    for b, (X_test, y_test, policy) in enumerate(test_loader):
        policy_list.append(policy)
        X_test = X_test.to(device)
        y_test = y_test.to(device)
        if b == max_tst_batch:
            break

        # Apply the model
        y_val = AlexNetmodel(X_test)
        y_val_list.append(y_val.data)
        # Tally the number of correct predictions
        predicted = torch.max(y_val.data, 1)[1] 
        tst_corr += (predicted == y_test).sum()

loss = criterion(y_val, y_test)
test_losses.append(loss)
test_correct.append(tst_corr)


My batch size is 100 for 15600 so I get a list of 156 tensors consisting of 100 policies each and another list of 156 tensors consisting of predictions.   Here is a print out:

image.png766×282 8.31 KB

I’m guessing I would need to unpack this list and the policy list and join them in a dataframe.  Then average the predictions by policy?