𝐿(𝑦, 𝑓𝜃 (𝑥)) = 𝐸(𝑥,𝑦)∼𝑝𝑑𝑎𝑡𝑎(𝑥,𝑦)[𝐿(𝑦, 𝑓𝜃 (𝑥))]

(27)
where 𝑝𝑑𝑎𝑡𝑎(𝑥, 𝑦) is the underlying data distribution.In semi-supervised learning, the loss function
is typically decomposed into two parts: a supervised loss term that is only defined on the labeled
data, and an unsupervised loss term that is defined on both labeled and unlabelled data. The
supervised loss term is calculated as follows:
1
𝑁𝐿

𝐿(𝑦, 𝑓𝜃 (𝑥))

∑︁

L𝑠𝑢𝑝 =

(28)

The unsupervised loss term leverages the unlabelled data to encourage the model to learn
meaningful representations that capture the underlying structure of the data. One common approach
is to use a regularization term that encourages the model to produce similar outputs for similar
input data. This can be achieved by minimizing the distance between the output of the model for
two similar input data points. One such regularization term is the entropy minimization term,
which can be expressed as:

(𝑥,𝑦)∈𝑋𝐿

𝑝𝜃 (𝑦 𝑗, 𝑥𝑖) log 𝑝𝜃 (𝑦 𝑗, 𝑥𝑖)

(29)
AReviewofDeepLearningTechniquesforSpeechProcessing