From some of the similar questions i did find that solution but when I tried to make it into a fixed filter I got the following errors
Heres the code
batch_size = 10
channels = 3
h, w = 24, 24
x = torch.randn(batch_size, channels, h, w)

# View approach
conv = nn.Conv2d(1, 1, 2, 2, bias=False)
with torch.no_grad():
    conv.weight = nn.Parameter(torch.tensor([[[[1, -1],
                                               [-1, 1]]]]))
output = conv(x.view(-1, 1, h, w)).view(batch_size, channels, h//2, w//2)

Error: Only Tensors of floating point and complex dtype can require gradients

When I write the values between 0 and 1 it works just fine.