Hi. Yes the number of images could be a reason depending on how you are loading them. To make the images to the same size, you can use a custom transform something like this:
# Transforms for DataLoader
class Resize(object):
    """ Resize Image and/or Masks """

    def __init__(self, height, width):
        """

        Args:
            height (int): resize to height
            width (int): resize to width
        """

        self.height = height
        self.width = width

    def __call__(self, sample):
        """

        Args:
            sample (dict): input image/mask pair

        Returns:
            dict: image/mask pair as ndarray
        """

        image, mask = sample['image'], sample['mask']
        dim = (self.width, self.height)
        image = image.resize(dim)
        mask = mask.resize(dim)

        return {'image': image,
                'mask': mask}

NOTE that the images in the above Resize transform are loaded using PIL.Image.
Don’t forget to convert the above images to tensor by either using the ToTensor transform as below:
# Convert Image from Numpy to Tensor
class ToTensor(object):
    """ Convert ndarray to Tensor """

    def __call__(self, sample):
        """

        Args:
            sample (dict): input image/mask pair

        Returns:
            torch Tensor: image/mask tensor
        """

        image, mask = sample['image'], sample['mask']

        image = np.asarray(image)
        mask = np.asarray(mask)

        if len(image.shape) == 3:
            image = image.transpose(2, 0, 1)

        if len(mask.shape) == 2:
            mask = mask.reshape((1,) + mask.shape)

        if len(image.shape) == 2:
            image = image.reshape((1,) + image.shape)

        return {'image': torch.from_numpy(np.array(image)),
                'mask': torch.from_numpy(np.array(mask))}

Then you can setup these transforms:
transforms.Compose([Resize(330, 422), ToTensor()])

The reason for writing these transforms instead of using the built in ones is due to image as feature and another image as label and we need to apply transforms to both so as to get a image/mask pair.
Hope this helps