Results are summarized in Fig. 5. First look at the scatter plots on the left. The x-axis represents
the degree of overﬁtting, which is computed as follows: denote LLtrain and LLval as the average
train and validation log-likelihood under the MLE estimation with Laplace smoothing (α = 1.0), the
degree of overﬁtting is deﬁned as (LLval − LLtrain)/LLval, which roughly captures how much the
dataset/model pair suffers from overﬁtting. The y-axis represents the improvement on the average
test set log-likelihood compared to Laplace smoothing. As demonstrated by the scatter plots, despite
a few outliers, both proposed regularization methods steadily improve the test set LL over various
datasets and PC structures, and the LL improvements are positively correlated with the degree of
overﬁtting. Furthermore, as shown by the last scatter plot and the histogram plot, when combining
data softening and entropy regularization, the LL improvement becomes much higher compared to
Tractable Regularization of Probabilistic Circuits