The GPU code could be faster if you vectorize (or batch) some operations. For example, for loops are generally slow because you’d launch a few cuda kernels per loop of the for loop, for every loop. Because your n_class and n_featuresare large, this means many cuda kernels get launched (and launching these things have overhead).
        k = 0
        for g in range(n_class):
            i = C[g]
            u = X[i][:,j].contiguous().view(-1,X[i][:,j].size()[0])
            k += (1 / i.size()[0]) * torch.sum(torch.mm(torch.t(u),u))
        k *= (1 / n)

I’m not really sure what you’re trying to do here but if you can replace the for loop purely with operations on the Variable, it’ll be faster to run with the GPU.