I think distribution could be implemented as a collection of functions as below:
class Distribution:

    @staticmethod
    def pdf(self):
        raise NotImplementedError

    @staticmethod
    def logpdf(self):
        raise NotImplementedError
    
    @staticmethod
    def cdf(self):
        raise NotImplementedError
    
    @staticmethod
    def qtl(self):
        raise NotImplementedError

    @staticmethod
    def rnd(self):
        raise NotImplementedError

class Gauss(Distribution):
    #scalar version
    @staticmethod
    def logpdf(x, mu, sigma):
        return -0.5 * (x-mu) * (x-mu)/(sigma*sigma) - 0.5 * torch.log(2*math.pi*sigma*sigma)


and it could be used like this:
Gauss.logpdf(x, mu, sigma)