readme: ---
  language:
  - multilingual
  license: apache-2.0
  inference: false
  tags:
  - youtube
  - video
  - pytorch
  
  ---
  
  # YouTube video semantic similarity model (NT = no transcripts)
  
  This YouTube video semantic similarity model was developed as part of the RegretsReporter research project at Mozilla Foundation. You can read more about the project [here](https://foundation.mozilla.org/en/youtube/user-controls/) and about the semantic similarity model [here](https://foundation.mozilla.org/en/blog/the-regretsreporter-user-controls-study-machine-learning-to-measure-semantic-similarity-of-youtube-videos/).
  
  You can also easily try this model with this [Spaces demo app](https://huggingface.co/spaces/mozilla-foundation/youtube_video_similarity). Just provide two YouTube video links and you can see how similar those two videos are according to the model. For your convenience, the demo also includes a few predefined video pair examples.
  
  ## Model description
  
  This model is custom PyTorch model for predicting whether a pair of YouTube videos are similar or not. The model does not take video data itself as an input but instead it relies on video metadata to save computing resources. The input for the model consists of video titles, descriptions, transcripts and YouTube channel-equality signal of video pairs. As illustrated below, the model includes three [cross-encoders](https://www.sbert.net/examples/applications/cross-encoder/README.html) for determining the similarity of each of the text components of the videos, which are then connected directly, along with a channel-equality signal into a single linear layer with a sigmoid output. The output is a similarity probability as follows:
  - If the output is close to 1, the model is very confident that the videos are similar
  - If the output is close to 0, the model is very confident that the videos are not similar
  - If the output is close to 0.5, the model is uncertain
  
  ![Model architecture](architecture.png)
  
  For pretrained cross-encoders, [mmarco-mMiniLMv2-L12-H384-v1](https://huggingface.co/cross-encoder/mmarco-mMiniLMv2-L12-H384-v1) was used to be further trained as part of this model.
  
  **Note**: sometimes YouTube videos lack transcripts so actually there are two different versions of this model trained: a model with trascripts (WT = with transcripts) and a model without transcripts (NT = no transcripts). This model is without transcripts and the model with transcripts is available [here](https://huggingface.co/mozilla-foundation/youtube_video_similarity_model_wt).
  
  **Note**: Possible model architecture enhancements are discussed a bit on [this blog post](https://foundation.mozilla.org/en/blog/the-regretsreporter-user-controls-study-machine-learning-to-measure-semantic-similarity-of-youtube-videos/) and some of the ideas were implemented and tried on experimental v2 version of the model which code is available on the RegretsReporter [GitHub repository](https://github.com/mozilla-extensions/regrets-reporter/tree/main/analysis/semsim). Based on the test set evaluation, the experimental v2 model didn't significantly improve the results. Thus, it was decided that more complex v2 model weights are not released at this time.
  
  ## Intended uses & limitations
  
  This model is intended to be used for analyzing whether a pair of YouTube videos are similar or not. We hope that this model will prove valuable to other researchers investigating YouTube.
  
  ### How to use
  
  As this model is a custom PyTorch model, not normal transformers model, you need to clone this model repository first. The repository contains model code in `RRUM` class (RRUM stands for RegretsReporter Unified Model) in `unifiedmodel.py` file. For loading the model from Hugging Face model hub, there also is a Hugging Face model wrapper named `YoutubeVideoSimilarityModel` in `huggingface_model_wrapper.py` file. Needed Python requirements are specified in `requirements.txt` file. To load the model, follow these steps:
  1. `git clone https://huggingface.co/mozilla-foundation/youtube_video_similarity_model_nt`
  2. `pip install -r requirements.txt`
  
  And finally load the model with the following example code:
  ```python
  from huggingface_model_wrapper import YoutubeVideoSimilarityModel
  model = YoutubeVideoSimilarityModel.from_pretrained('mozilla-foundation/youtube_video_similarity_model_nt')
  ```
  
  For loading and preprocessing input data into correct format, the `unifiedmodel.py` file also contains a `RRUMDataset` class. To use the loaded model for predicting video pair similarity, you can use the following example code:
  ```python
  import torch
  import pandas as pd
  from torch.utils.data import DataLoader
  from unifiedmodel import RRUMDataset
  
  video1_channel = "Mozilla"
  video1_title = "YouTube Regrets"
  video1_description = "Are your YouTube recommendations sometimes lies? Conspiracy theories? Or just weird as hell?\n\n\nYou’re not alone. That’s why Mozilla and 37,380 YouTube users conducted a study to better understand harmful YouTube recommend