I am reading through the ‘Extending PyTorch’ 25 that implements a new nn.Module and uses the function that has both forward() and backward() implemented. My question is if I want to define a new module that just adds a random weight to the input tensor and I define it like this:
class CustomLayer(nn.Module):
    def __init__(self, input_dim):
        super(CustomLayer, self).__init__()
        self.input_dim = input_dim
        self.weight = nn.Parameter(torch.Tensor(input_dim))
        self.weight.data.uniform_(-0.05, 0.05)

    def forward(self, input):
        return torch.add(input, self.weight)

Then do I need to implement a backward() as well? Because in the forward method, I am not using a Function, but just an already implemented operation in PyTorch.
Does having an nn.Parameter() in my contructor requires me to write a backward() function? Do they get updated without havin a backward()?
To my understanding if only you implement a custom function and you want to somehow manipulate the backward operations, then you need to write a backward() function. So for example, the module in the tutorial can be simply written as:
class Linear(nn.Module):
    def __init__(self, input_features, output_features, bias=True):
        super(Linear, self).__init__()
        self.input_features = input_features
        self.output_features = output_features
        self.weight = nn.Parameter(torch.Tensor(output_features, input_features))
        if bias:
            self.bias = nn.Parameter(torch.Tensor(output_features))
        else:
            self.register_parameter('bias', None)

        self.weight.data.uniform_(-0.1, 0.1)
        if self.bias is not None:
            self.bias.data.uniform_(-0.1, 0.1)

    def forward(self, input):
        return torch.add(torch.matmul(input, self.weight), self.bias)

without a backward() method!