log-parameters that correspond to a stationary point of the objective.2 In its main loop (lines 4-10),
the algorithm alternates between two procedures: (i) compute the entropy of the distribution encoded
by every node w.r.t. the current parameters (line 5),3 and (ii) update PC parameters with regard to the
computed entropies (lines 6-10). Speciﬁcally, in the parameter update phase (i.e., the second phase),
the algorithm traverses every sum unit n in preorder and updates its child parameters by maximizing
the entropy regularization objective (LLent(θ;D, τ )) with all other parameters ﬁxed. This is done by
solving the set of equations in Eq. (5) using Newton’s method (lines 7-8).4 In addition to the child
nodes’ entropy computed in the ﬁrst phase, Eq. (5) uses the top-down probability of every unit n (i.e.,
node_prob[n]), which is progressively updated in lines 9-10.
Theorem 4. Alg. 3 converges monotonically to a stationary point of LLent(θ;D, τ ) (Eq. (2)).
Tractable Regularization of Probabilistic Circuits