How do you set your seed? There’s a few things that you should set to a fixed value to be sure. I always use the following, which has worked consistently.
    def set_seed(seed: Optional[int]):
        """ Set all seeds to make results reproducible (deterministic mode).
             When seed is None, disables deterministic mode. """
        if seed is not None:
            torch.manual_seed(seed)
            torch.cuda.manual_seed_all(seed)
            torch.backends.cudnn.deterministic = True
            torch.backends.cudnn.benchmark = False
            np.random.seed(seed)
            random.seed(seed)
            os.environ['PYTHONHASHSEED'] = str(seed)

But even then, I don’th think variance is to be expected in inference mode. Not sure what is happening here.