Trainable Variable in Loss Function and Matrix Multiplication
Hi, could you check the example with trainable-variable? I was thinking about two cases: implementing a Linear layer without torch.nn. implementing the Loss layer with learable parameter (ex. Center Loss where mean-center is trainable parameters). Such stuff are easy in TesnotFlow, but I’m not sure if I coded it correctly in PyTorch Could you check if the idea of using PyTorch here is correct? Or I should change sth here.
melgor
Hi! There are some issues with your first script: you need to define linear1 and linear2 as nn.Parameter instead of Variable, if not model.parameters() won’t see them just for information, nn.Linear implements it as torch.mm(input, weight.t() 61, as in torch7 BTW you might want to divide the loss by the batch size, that’s what’s the default in pytorch
fmassa
I’ve implemented the same loss, however, the centers learned are nearly the same. I don’t know where is the problem.
waitwaitforget
jxgu1016
A more easily implenment of center loss. <SCODE>class CenterLoss(nn.Module):

    def __init__(self,center_num,feature_dim):
        super(CenterLoss,self).__init__()
        self.center_num = center_num
        self.feature_dim = feature_dim
        self.center_features = nn.Parameter(torch.Tensor(self.center_num,self.feature_dim))
        nn.init.normal(self.center_features.data,mean=0,std=0.1)

    def forward(self,x,label):
        B = x.size()[0]
        center = torch.index_select(self.center_features,0,label)
        diff = x-center
        loss = diff.pow(2).sum() / B
        return loss
<ECODE>
CKboss
I had the same issue. My parameters learned are the same. Did you solve it and how? Thanks!
lelegu