I started putting something online at https://github.com/stepelu/ptstat 225, for now just some statistical distributions and a small number of related functions.
I’ll implement the Gamma, Beta and Dirichlet distributions too next week, sampling aside that would need a C/CUDA implementation (and it’s not trivial to do CUDA-efficiently and in a numerically stable way).
I am also working on MCMC: Metropolis-RW, Langevin-RW, HMC, NUTS.
Contributions are welcome, for instance:

not yet implemented statistical distributions (let me know if you plan to work on any of the above)
add missing kl-divergence implementations
add mean and variance methods

Is there any timeline for issue #274? It would allow me to remove all these Variable() calls and facilitate the MCMC code I’m writing too.