  We study the Markov semigroups for two important algorithms from machine
learning: stochastic gradient descent (SGD) and online principal component
analysis (PCA). We investigate the effects of small jumps on the properties of
the semi-groups. Properties including regularity preserving, $L^{\infty}$
contraction are discussed. These semigroups are the dual of the semigroups for
evolution of probability, while the latter are $L^{1}$ contracting and
positivity preserving. Using these properties, we show that stochastic
differential equations (SDEs) in $\mathbb{R}^d$ (on the sphere
$\mathbb{S}^{d-1}$) can be used to approximate SGD (online PCA) weakly. These
SDEs may be used to provide some insights of the behaviors of these algorithms.

Probability