i made a function code that makes the zero weight fixed to zero while training by making zero weight’s grad to zero.
def fix_tensor_data(model.parameters()):
    for t in model.parameters():
        if t.grad is not None:
            t_copy=t.data.abs().clone()
            mask = t_copy.gt(0).float().to(t.grad.device)
            t.grad.mul_(mask)  
            Lgt=torch.stack([t.grad])             
    return Lgt   

and i check that optimizer’s grad have a zero grad after applying this function. but zero weights are changed when i run ‘optimizer.step()’
the optimizer’s code used in weight updating  below
(in DARTS which is autoML model)
  optimizer = torch.optim.SGD(
      model.parameters(),
      args.learning_rate,
      momentum=args.momentum,
      weight_decay=args.weight_decay)

i added a ‘fix_tensor_data’ before optimizer.step() and checked that
zero grads are in optimizer’s param after applying ‘fix_tensor_data’.
    fix_tensor_data(model.parameters())
     
    print(optimizer.param_groups[0]["params"][0].grad)
    optimizer.step() 

but when i run the optimizer.step() and print model.parameters(), there are no zero weight in parameter tensors.
so i find step() library in pytorch but i can’t get it…
what is the problem of my code?..
i really need a help… thank u for reading my question!