Your Python code is wrong, as you are creating a new .weights attribute while you would like to replace the .weight parameter.
Change it to:
x_kernel = torch.randn(3, 3)
rgb_image = torch.randn(1, 3, 24, 24)

conv = torch.nn.Conv2d(in_channels=3, out_channels=3, groups=1, kernel_size= (3,3), padding=3//2, bias=False)
conv.weight = nn.Parameter(x_kernel.unsqueeze(0).unsqueeze(0).repeat(1,3,1,1))
out = conv(rgb_image)
print(out.shape)
> torch.Size([1, 1, 24, 24])

and you will get the same output size as in libtorch.