@junpenglao did you start something concerning distributions? I am starting a project involving stochastic networks, so I am going implement a kind of distribution library with tools for estimation (assuming a tensor follows a distribution) and sampling.
My idea is to create one objects distrib.Distribution (e.g. distrib.Bernouilli() ) and two funcions:
distrib.estimation(tensor x, distribution A) = estimation of parameters of distribution A from sample x
distrib.sample(distribution A) = tensor x ~ A
For exemple, this would be an implementation of a RBM with nv visible units and nh hiden:
W = torch.rand(nv, nh)); a = torch.rand(nh)); b = torch.rand(nv));
net_forward = nn.Sequential(nn.Linear(wieght=W, bias=a), nn.Sigmoid())
net_backward = nn.Sequential(nn.Linear(wieght=W, bias=b), nn.SoftMax())

# training:
for input in training_set:
    v1 = input # from training set

    x = net_forward(v1)
    H1 = distrib.estimation(x, distrib.Bernouilli())
    h1 = distrib.sample(H1)

    y = net_backward(h)
    V2 = distrib.estimation(y, distrib.Multinomial())
    v2 = distrib.sample(V2)

    x = net_forward(v2)
    H2 =distrib. estimation(x, distrib.Bernouilli())
    h2 = distrib.sample(H2)

    # gradient descent step:
    W -= lr * (v1.h1 - v2.h2)
    a -= lr * (h1 - h2)
    b -= lr * (v1 - v2)