4.4.2 Contrastive Models
The technique involves training a model to differentiate between similar and dissimilar pairs of data
samples, which helps the model acquire valuable representations that can be utilized for various
tasks, as shown on Figure 12. The fundamental principle of contrastive learning is to generate
positive and negative pairs of training samples based on the comprehension of the data. The model
must learn a function that assigns high similarity scores to two positive samples and low similarity
scores to two negative samples. Therefore, generating appropriate samples is crucial for ensuring
that the model comprehends the fundamental features and structures of the data. Table 3 outlines
popular contrastive self-supervised models used for different speech-processing tasks. We discuss
Wav2Vec 2.0 since it has achieved state-of-the-art results in different downstream tasks.
AReviewofDeepLearningTechniquesforSpeechProcessing