Hi,
Could it be that the first gradient step pushes everything on the flat side of the hinge and so it just returns 0?
Also few question:

Why so many .cuda() calls? You should not need them if everything is already on the GPU. If everything is on the CPU, you might not want to shuffle things around that much for perf reasons.

torch.mul(torch.ones(n_class).cuda(), y_out) is the same as y_out right?

anti_y = torch.ones(n_class).cuda() - y.cuda() can be done as anti_y = 1 - y.cuda() to make it simpler to read and more efficient

torch.zeros(n_class).cuda() can be replaced by torch.zeros(n_class, device="cuda") if you want.

torch.max(loss.cuda(), torch.zeros(n_class).cuda()) -> torch.threshold(loss, 0, 0)