Your code snippet will calculate the accuracy for an imbalanced binary classification use case for class5 only.
Note that by selecting class5, you will get a lot of “non-class5” outputs, which will create a high True Negative score.
E.g. if your current segmentation output contains only a single pixel with class5, which is also wrongly classified, you’ll still get a very high accuracy as shown in this example:
N, H, W = 2, 20, 20
nb_classes = 6
output = torch.randn(N, nb_classes, H, W)
# Predict only a single pixel as class5
output[:, 5] = -1000000
output[0, 5, 0, 0] = 100000

preds = torch.argmax(output, 1)

# Create only single target as class5
labels = torch.randint(0, nb_classes-1, (N, H, W))
labels[0, 1, 1] = 5

labels1 = labels.eq(5).float()
preds1 = preds.eq(5).float()
print(torch.sum(preds1 == labels1).float() / labels1.nelement())
> tensor(0.9975)