My sincerest apologies for the goof-up. I don’t know what I was thinking. This is how I initialised weights of my convolution layer:
conv1 = nn.Conv2d(1, 1, kernel_size=2)
with torch.no_grad():
  conv1.weight.data = torch.tensor([[-0.8423,  0.3778],[-3.1070, -2.6518]]) 

And then, I just defined the model like this:
class prune_net(nn.Module):
    def __init__(self):
        super(prune_net, self).__init__()
        self.conv1 = conv1
        self.pool = nn.MaxPool2d(2, 2)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        return x