No, I do want to learn both weights and the threshold. Currently I have to do it manually:
loss = loss + 0.001 * model.w_max ** 2
optimizer.zero_grad()
loss.backward()
w_max_grad = torch.sum(model.conv1.weight.grad[model.conv1.weight >= model.w_max])  
model.w_max.grad.data += w_max1_grad
optimizer.step()

Note that I force L2 penalty on the threshold growth, so I have to add the clipped weight gradients to the L2 loss gradient for the threshold before I update its value.
This works, however when I do the same thing for activation clipping, I don’t need to do anything manually - the gradients are accumulated correctly in the backward pass.