Once you’ve loaded each image and transformed it to a tensor in the shape [channels, height, width], you could concatenate it with torch.cat as seen here:
gray = torch.randn(1, 24, 24)
rgb = torch.randn(3, 24, 24)
x = torch.cat((gray, rgb), dim=0)
print(x.shape)
> torch.Size([4, 24, 24])