Hello, I want to define a new loss function. Specifically, with the L1 norm applied on the weight of FC layer, my aim is to learn sparse feature. The code is following:
    params = model.state_dict()
    p = params['module.fc.weight']
    loss  = nn.CrossEntropyLoss()(output, target_var) + lambda*L1Loss(p)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

here is my loss function
def L1loss(weight)
   loss = torch,norm(weight, p=1)
   return loss

Obviously, it does work. The L1 loss dosen’t decrease, but the CrossEntropyLoss seems well.
So, How to combine them? or my definition on L1 loss is wrong?