Hi thank you the custom leaky. I was wondering. if i trained the negative slope value here, where please correct me if i am wrong. I need to pass the gradient required for the slope in backward propagation as i did below after calculating the gradient for slope
@staticmethod
def forward(ctx, input,negative_slope):
output = input.clamp(min=0)+input.clamp(max=0)*negative_slope
ctx.save_for_backward(input)
ctx.slope = negative_slope
return output
@staticmethod
def backward(ctx, grad_output):
input, = ctx.saved_tensors
slope = ctx.slope
grad_input = grad_output.clone()
grad_input = grad_input * (input > 0).float() + grad_input * (input < 0).float() * slope
return grad_input, negative_slope_gradient

how can i get the updated negative slope after training and also, how can i check if it is training simultaneously. I, also understand we need to do negative_slope.requires_grad = True. Any help is appreciated. Thank you