You can follow the tutorial here 31. The derivatives for LeakyReLU when x>0 is 1 and -NEGATIVE_SLOPE when x<=0. Like what @nthn_clmnt said, the argument self shouldn’t be named “self” becuase it is very confusing, it is actually a “context” object that holds information. When apply is called, both forward's and backward's first argument will be the context object, followed by other arguments, so input is the actual input you passed.