readme: ---
  license: cc-by-nc-nd-4.0
  metrics:
  - accuracy
  tags:
  - generated_from_trainer
  - Text Generation
  - Primary Sequence Prediction
  model-index:
  - name: protgpt2-finetuned-sarscov2-rbd
    results: []
  ---
  
  # Model Card for `protgpt2-finetuned-sarscov2-rbd`
  
  This model is a fine-tuned version of [nferruz/ProtGPT2](https://huggingface.co/nferruz/ProtGPT2) on sequences from the NCBI Virus Data Portal.
  
  It achieves the following results on the evaluation set:
  - Loss: 1.1674
  - Accuracy: 0.8883
  
  ## Model description
  
  This model is a fine-tuned checkpoint of
  [ProtGPT2](https://huggingface.co/nferruz/ProtGPT2), which was originally
  trained on the UniRef50 (version 2021_04) database. For a detailed overview
  of the original model configuration and architecture, please see the linked
  model card, or refer to the ProtGPT2 publication.
  
  The model was finetuned on data from the SARS-CoV-2 Spike (surface glycoprotein)
  receptor binding domain (RBD).
  
  A repository with the training scripts, train and test data partitions, as well
  as evaluation code is available on GitHub at
  (https://github.com/rahuldhodapkar/PredictSARSVariants).
  
  ## Intended uses & limitations
  
  This model is intended to generate synthetic SARS-CoV-2 surface glycoprotein
  (a.k.a. spike protein) sequences for the purpose of identifying meaningful
  variants for characterization either experimentally or through other
  *in silico* tools.  These variants may be used to drive vaccine develop to
  protect against never-before-seen point mutants that are probable in the future.
  
  As this model is based on the original ProtGPT2 model, it is subject to many
  of the same limitations as the base model.  Any biases present in the UniRef50
  dataset will also be present in the model, which may include nonuniform skew
  of peptides sampled across different taxonomic clades.  These limitations
  should be considered when interpreting the output of this model.
  
  ## Training and evaluation data
  
  SARS-CoV-2 spike protein sequences were obtained from the NIH Sars-CoV-2 Data Hub
  accessible at 
  
      https://www.ncbi.nlm.nih.gov/labs/virus/vssi/
  
  Note that the reference sequence for the surface glycoprotein can be found at:
  
      https://www.ncbi.nlm.nih.gov/protein/1791269090
  
  As the loaded ProtGPT2 model was pretrained on the
  UniRef50 (version 2021_04) dataset, it cannot have contained sequencing
  data that was generated after that date.  Evaluations will be conducted using
  SARS-CoV-2 sequences generated on or after May 2021.
  
  ## Training procedure
  
  ### Training hyperparameters
  
  The following hyperparameters were used during training:
  - learning_rate: 1e-05
  - train_batch_size: 16
  - eval_batch_size: 16
  - seed: 42
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  - lr_scheduler_type: linear
  - num_epochs: 3.0
  
  ### Framework versions
  
  - Transformers 4.26.0.dev0
  - Pytorch 1.11.0
  - Datasets 2.8.0
  - Tokenizers 0.13.2
  
  