## Refinement of genetic variants needs attention

Omar Abdelwahab 1,2,3,4 , Davoud Torkamaneh 1,2,3,4 *

- 1 Département de Phytologie, Université Laval, Québec, Canada
- 2 Institut de Biologie Intégrative et des Systèmes (IBIS), Université Laval, Québec, Canada
- 3 Centre de recherche et d'innovation sur les végétaux (CRIV), Université Laval, Québec, Canada
- 4 Institut intelligence et données (IID), Université Laval, Québec, Canada
- * Corresponding author: davoud.torkamaneh.1@ulaval.ca

## Abstract

Variant calling refinement is crucial for distinguishing true genetic variants from technical artifacts in high-throughput sequencing data. Manual review is time-consuming while heuristic filtering often lacks optimal solutions. Traditional variant calling methods often struggle with accuracy, especially in regions of low read coverage, leading to false-positive or false-negative calls. Here, we introduce VariantTransformer, a Transformer-based deep learning model, designed to automate variant calling refinement directly from VCF files in low-coverage data (10-15X). VariantTransformer, trained on two million variants, including SNPs and short InDels, from lowcoverage  sequencing  data,  achieved  an  accuracy  of  89.26%  and  a  ROC AUC  of  0.88.  When integrated into conventional variant calling pipelines, VariantTransformer outperformed traditional heuristic filters and approached the performance of state-of-the-art AI-based variant callers like DeepVariant. Comparative analysis demonstrated VariantTransformer's superiority in functionality,  variant  type  coverage,  training  size,  and  input  data  type.  VariantTransformer represents  a  significant  advancement  in  variant  calling  refinement  for  low-coverage  genomic studies.

## Introduction

Genetic variants are considered the backbone for identifying mutations within populations and are fundamental to genetic screening tools  1 . Variant calling is the process of identifying differences between  an  individual's  genome  and  a  reference  genome,  encompassing  single  nucleotide polymorphisms  (SNPs),  small  insertions  and  deletions  (InDels),  and  structural  variations   . 2 However,  the  raw  output  from  high-throughput  sequencing  data  often  requires  refinement  to distinguish true genetic variants from technical artifacts. Variant calling refinement, a critical postprocessing stage following variant calling, is aimed at filtering out erroneous calls and enhancing the accuracy of detected variants. While variant calling provides an initial catalog of genomic variations, refinement is essential to minimize false positives and false negatives, ensuring that downstream analyses are built upon a foundation of reliable genetic data 3,4 .

Typically,  variant  refinement  involves  heuristic  filtering  and/or  manual  inspection.  Heuristic filtering entails establishing project-specific thresholds for key metrics such as read depth, variant allele  fraction  (V AF),  base  quality,  read  quality, and  mapping quality  scores    .  Manual  review, 5 while time-intensive, enhances confidence in specific variants by uncovering patterns typically overlooked by conventional variant callers through direct visual inspections of the variants using genomic viewers like Integrative Genomic Viewer (IGV)   6,7 . Although heuristic filtering offers speed, it often fails to provide the optimal solution if it does not align with user-defined thresholds 8-11 . Strict heuristic filtering leads to more false negatives, while relaxed filtering results in a higher number of false positives. Despite its importance, the refinement process in variant calling is often underdeveloped and lacks comprehensive representation in genomic workflows.

With the emergence of artificial intelligence (AI), new models have been introduced to enhance the process of variant calling refinement 12-14 . These early efforts have shown potential in utilizing machine and deep learning (ML and DL) techniques to improve the precision of variant calling. However, they often focused on limited datasets  predominantly associated  with  cancer-related diseases  and  are  derived  from  high-coverage  sequencing  data,  restricting  their  utility  across  a broader spectrum of genomic studies 15 .

Transformers,  a  revolutionary  class  of  deep  learning  models originally  developed  for  natural language processing (NLP), have demonstrated exceptional capabilities in identifying complex patterns and dependencies in sequential data 16 . Their unique architecture, which includes 'self-

attention' and 'feed-forward neural network' layers allows for dynamic learning of correlations among features, thereby enhancing classification tasks 16 . Given their adaptability, Transformers are ideally suited to address the challenges of variant calling refinement, enabling the thorough analysis  of  extensive  genomic  datasets  and  facilitating  the  extraction  of  high-quality  genomic variations.

In  this  study,  we  introduce  VariantTransformer,  a  model  that  automates  the  variant  calling refinement  process.  This  model  is  capable  of  directly  processing  variant  calling  format  files (VCFs) and can be seamlessly integrated into standard variant calling pipelines such as BCFTools 17 and GATK  18 . Specifically designed for low-coverage sequencing data, VariantTransformer uses a dataset of 2 million variants, the largest utilized for such a purpose to date. We treat each variant in the VCF as a distinct 'sentence', thus conceptualizing the refinement process as a 'sentence classification  problem'  where  the  goal  is  to  categorize  each  variant  as  'Pass'  or  'Fail'.  This classification is then used to update the 'Filter' column in the VCF, thereby enhancing the model's applicability and utility in genomic studies.

## Methods

## Sequencing data

The  FASTQ files  containing  sequencing  data,  generated  on  an  Illumina  HiSeq2500,  for  three samples (HG003, HG006, and HG007) with sequence coverages of 10.5X, 13.6X, and 12.6X respectively, were procured from the Genome in a Bottle (GIAB) Consortium  19 , accessed via the NIST GIAB FTP site (https://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/data/). We used SAMtools  20  to determine the coverage of samples, utilizing the ' -a ' option, to consider all positions within the reads.  The  alignment  of  the  raw  FASTQ  files  to  the  human  reference  genome  GRCh38 (GCA\_000001405.15\_GRCh38\_no\_alt\_analysis\_set.fna)  was  performed  using  Sentieon  BWAMEM  21 .

## Training and testing Data

Variant  calling  on  the  aligned  BAM  files  was  performed  with  GATK4  HaplotypeCaller 18 and BCFTools  17 . To ensure accurate variant classification, we used the latest GIAB truth sets v4.2.1 22 to  update  the  'FILTER'  column  in  the VCFs;  variants  matching  the  truth  sets  were  labeled 'PASS' while all others were labeled as 'FAIL'.

During preprocessing, we simplified the dataset by removing non-essential columns ('Chrom,' 'POS,'  'REF,'  and  'ALT')  and  consolidating  the  remaining  data  into  a  single  column.  This restructuring  facilitated  the  transformation  of  the  dataset  into  a  sentence  classification  format, where the merged column represented the 'sentences' and the 'FILTER' column the target labels.

For training purposes, we merged the updated VCFs (generated from GATK4 and BCFTools) from the sample HG003. To reduce computational cost in the training process, we randomly selected 2 million  variants  from  the  merged  VCF  for  initial  model  training  and  validation.  Table  1 demonstrates the number of variants generated at each step. The selected dataset was then split into 60% for training and 40% for validation, with the remaining variants of the HG003 and the other VCFs from the other samples being used for further testing.

Table 1 . Breakdown of "PASS" and "FAIL" variants from sample HG003, processed with GATK4 and BCFTools.

| HG003 sample                | PASS      | FAIL      | Total     |
|-----------------------------|-----------|-----------|-----------|
| BCFTools                    | 3,714,910 | 708,823   | 4,423,733 |
| GATK4                       | 3,668,708 | 964,363   | 4,633,071 |
| Total_merged_file           | 7,383,618 | 1,673,186 | 9,056,804 |
| Training and validation set | 1,630,866 | 369,134   | 2,000,000 |

In the testing process, we used two other samples (HG006 and HG007) (Table 2). We tested in batches of 10,000 to generate probabilities for further model performance analyses.

Table 2 . Variant counts obtained from samples HG006 and HG007 via GATK4 and BCFTools, detailing "PASS" and "FAIL" variants.

| Sample   |   Coverage (X) | Variant Caller   | PASS      | FAIL      | Total     |
|----------|----------------|------------------|-----------|-----------|-----------|
| HG006    |           13.6 | BCFTools         | 3,694,662 | 826,173   | 4,520,835 |
| HG006    |           13.6 | GATK4            | 3,666,347 | 1,066,280 | 4,732,627 |
| HG007    |           12.6 | BCFTools         | 3,685,056 | 847,135   | 4,532,191 |
| HG007    |           12.6 | GATK4            | 3,650,142 | 1,042,385 | 4,692,527 |

## Model development and analysis

A DL model was developed based on the Transformers architecture 16 to  automate the variant calling refinement process. We used the 'BertForSequenceClassification' model from Hugging Face 23 while tuning some of the parameters in the configuration and tokenization steps to achieve better performance (Supplementary Table S1). This selection was made due to the model's aptitude for  handling  sequence-based  data  with  complex  dependencies.  Parameters  were  finely  tuned, including the addition of variant calling specific vocabulary to the BertTokenizer and adjustments to several hyperparameters (Supplementary Table S3) to optimize the model's predictive accuracy and computational efficiency. For the rest of the parameters, we used the default values mentioned in the Hugging Face documentation 23 . The model trained over 21 epochs with a batch size of 1,300 using the AdamW optimizer  24 , focusing on balancing performance and resource utilization.

## Evaluation Metrics

To thoroughly evaluate model performance, we employed several accuracy metrics including the AUC (Area Under The  Curve)  ROC  (Receiver  Operating  Characteristics)  curve 25 ,  Matthews correlation coefficient (MCC) 26-29 , accuracy, precision, recall, and F1 score 30 . For further model evaluation, we reported MCC, also known as the phi coefficient, where a coefficient of +1 indicates an ideal prediction, 0 signifies an average random prediction, and -1 denotes a reverse prediction 27,29 . All accuracy metrics were generated using scikit-learn 31 . Moreover, we compared the model performance  to  default  filtering  parameters  of  conventional  variant  callers  (BCFTools  and GATK4), AI-based variant caller (DeepVariant  32 ), and AI-based tool for refinement of somatic variant  calling  (DeepSVR 14 )  considering  both  PASS  and  FAIL  variants.  For  BCFTools  and GATK4, we integrated VariantTransformer into each pipeline and compared the performance of the model against default filters. For GATK4, the default filters were QD &lt; 2.0, FS &gt; 60.0, MQ &lt; 40.0, SOR &gt; 4.0, MQRankSum &lt; -12.5, and ReadPosRankSum &lt; -8.0. For BCFTools, we applied only the QUAL&gt;=20 filter. As for DeepVariant, we compared the performance of each modelintegrated  conventional  variant  caller  against  DeepVariant.  DeepSVR  was  compared  against VariantTransformer  in  terms  of  data  preparation,  multiple  accuracy  metrics,  computational complexity, and user experience. All plots were generated using the MatPlotlib library 33 or ggplot2 34 .

## Data availability

The complete dataset utilized in this study is accessible via the NIST GIAB FTP site: https://ftptrace.ncbi.nlm.nih.gov/giab/ftp/data/. All the resources related to the analyses, including preprocessing code, training and testing code, and the trained model can be found in the GitHub repository: https://github.com/Omar-Abd-Elwahab/VCF\_filter. This repository also offers guidance on using the model for filtering VCFs, adapting it to other sequencing technologies, or integrating it into different variant calling pipelines.

## Results

## Model development

VariantTransformer was developed using a dataset of 2 million variants, including both NPs and InDels,  sourced  from  the  GIAB  sample  HG003.  The  variants  were  called  using  GATK4  and BCFTools. Of the two million variants, 1,630,866 matched the latest GIAB truth sets and were classified as 'PASS', while the remaining 369,134 were classified as 'FAIL'. To avoid overfitting, the data was randomly split into 60% for training and 40% for validation. The model achieved an accuracy of 89.26% and ROC AUC score of 0.88, demonstrating robust performance (Figure 1a).

Figure1. a) ROC AUC curve for the trained model. b) VariantTransformer's average performance in  refining  variants  called  from  the  GIAB  samples  HG006  and  HG007,  benchmarked  against default filtering and DeepVariant.

![image](https://ac.marcodsn.me/arxiv-markdown-images/2408.00659v1-figure-f54f68ca-c3d3-4d4d-90c9-ca0a629e9f19.jpg)

## Model testing and evaluation

To  test  VariantTransformer,  we  integrated  it  into  two  conventional  variant  calling  pipelines (BCFTools and GATK4). We processed the two GIAB samples HG006 and HG007 through these pipelines  and  subsequently  applied  VariantTransformer  to  refine  the  variant  calls.  The  model significantly outperformed the default threshold-based filtering, achieving an overall accuracy of 87%, compared to 78% for GATK4 and 83% for BCFTools, thus approaching the performance of the AI-based variant caller, DeepVariant, which has an accuracy of 88% (Figure 1b).

During  the  testing  phase,  batches  of  10,000  variants  were  processed  to  generate  performance metrics  such  as  MCC  scores,  ROC AUC  scores,  accuracy,  precision,  recall,  and  F1score.  The aggregated results, outlined in Table 4, provide a comprehensive evaluation of VariantTransformer's efficacy when integrated into the aforementioned variant calling pipelines.

Table 4 .  Performance of VariantTransformer across different metrics in the variant calling pipelines for samples HG006 and HG007.

| Sample   | Coverage (X)   | Variant Caller   | Accuracy*   |   Refined Accuracy** | Precision   |   Recall | F1score   | ROCAUC   | MCC     |
|----------|----------------|------------------|-------------|----------------------|-------------|----------|-----------|----------|---------|
| HG006    | BCFTools       | 83.31%           | 87.03%      |              0.85957 | 0.87027     |  0.85845 | 0.86413   |          | 0.51194 |
| HG006    | 13.6 GATK4     | 77.50%           | 87.38%      |              0.87004 | 0.87375     |  0.86385 |           | 0.88876  | 0.60805 |
| 12.6     | BCFTools       | 82.63%           | 86.68%      |              0.85635 | 0.86678     |  0.85197 | 0.83063   | 0.50117  |         |
| 12.6     | GATK4          | 77.82%           | 86.94%      |              0.86646 |             |  0.86944 | 0.85692   | 0.85605  | 0.58638 |

* Default filtering

** After VariantTransformer refinement

## Comparative analysis: assessing performance against existing pipelines and models

In contrast to most performance analysis studies that use the hap.py tool 35 and focus solely on variants  passing  the  filters,  our  study,  comprehensively  assessed  all  variants,  including  those identified as 'FAIL'. This comprehensive approach considers all four parameters: true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN).

The performance of VariantTransformer was compared with BCFTools and GATK4 pipelines, and with DeepVariant, focusing on two GIAB samples (HG006 and HG007) with coverage of 13.6X and 12.6X (Table 5). For sample HG006  using BCFTools, from 4,520,835 variants, VariantTransformer identified  3,993,987  as  'PASS'  with  an  accuracy  of  87.03%. When  using GATK4 on the same sample, the model called 4,027,338 as 'PASS' with an accuracy of 87.38%. For sample HG007 using BCFTools, VariantTransformer identified 4,044,040 as 'PASS' with an accuracy of 86.68%. When using GATK4 on HG007, the model called 4,060,472 as 'PASS' with an  accuracy  of  86.94%.  Comparatively,  DeepVariant  demonstrated  slightly  higher  overall accuracy,  with  87.95%  for  HG006  and  87.78%  for  HG007.  However,  the  number  of  FP  was notably higher for DeepVariant compared to VariantTransformer, highlighting our model's balance between accuracy and precision.

Table 5. Detailed comparison of VariantTransformer's performance when integrated with GATK4 and BCFTools against DeepVariant

| Sample   | Coverage (X)   | Variant Caller   | Input*    | PASS**    | TP        | FP      | FN      | TN        | Accuracy   |
|----------|----------------|------------------|-----------|-----------|-----------|---------|---------|-----------|------------|
| HG006    |                | BCFTools         | 4,520,835 | 3,993,987 | 3,551,084 | 442,903 | 143,576 | 383,270   | 87.03%     |
|          | 13.6           | GATK4            | 4,732,627 | 4,027,338 | 3,548,098 | 479,240 | 118,249 | 587,039   | 87.38%     |
|          |                | DeepVariant      | 6,643,643 | 4,213,736 | 3,593,719 | 620,018 | 180,668 | 2,249,241 | 87.95%     |
|          |                | BCFTools         | 4,532,191 | 4,044,040 | 3,562,660 | 481,380 | 122,394 | 365,755   | 86.68%     |
| HG007    | 12.6           | GATK4            | 4,692,527 | 4,060,472 | 3,548,979 | 511,493 | 101,163 | 530,891   | 86.94%     |
|          |                | DeepVariant      | 6,716,326 | 4,247,514 | 3,597,227 | 650,288 | 170,251 | 2,298,563 | 87.78%     |

* Total called variants

** Identified as PASS by VariantTransformer in case of BCFTools and GATK4, or Identified as PASS by DeepVariant filter.

## Discussion

The  development  and  validation  of  the  Transformer-based  VariantTransformer  model,  for  the refinement of variant calling in low-coverage genomic data, represent a significant advance in leveraging  AI  technologies  in  genomics.  This  model  demonstrates  an  impressive  variant refinement accuracy of 89.26%, which not only outperforms conventional refinement methods but also  aligns  closely  with  contemporary  AI-based  tools.  The  higher  prediction  performance demonstrates that this automated strategy has the potential to substitute the conventional thresholdbased filtering methods, particularly in contexts involving low-coverage genomic data.

VariantTransformer's  effectiveness  in  processing -a  technique  borrowed  from  NLP   16 -highlights  its  ability  to  handle  the  complex  patterns  inherent  in  genomic  sequence  data.  By interpreting these patterns, the model distinguishes true genetic variants from technical artifacts 36 with high efficiency, achieving ROC AUC scores that affirm its capacity to differentiate variant classes across all thresholds, thus enhancing its utility in varied analytical scenarios.

The model's success can be attributed to several key factors. First, the use of an extensive dataset comprising 2 million variants, including both SNPs and short InDels, provided a robust foundation for both training and validation. Second, modifications to the default BERT model parameters, specifically, reductions in hidden size and the number of attention heads, have tailored the model to handle the specific complexity of genomic data while optimizing computational efficiency 16 . Finally, the MCC values highlight the model's balanced accuracy, considering both positive and negative classes, which is essential for applications in genomic studies where both sensitivity and specificity are critical 28 . The ROC AUC scores further affirm the model's exceptional capability to distinguish between the variant classes across all thresholds, emphasizing its effectiveness in various scenarios 37 .

When comparing VariantTransformer to DeepVariant, it is noteworthy that despite DeepVariant's slightly higher overall accuracy in some cases (87.95% for HG006 and 87.78% for HG007 with a difference of less than 1% when compared to VariantTransformer), it also presented a significantly higher rate of FPs for both samples. For sample HG006, DeepVariant had approximately 40% more FPs than BCFTools and approximately 29.38% more FPs than GATK4. For sample HG007, DeepVariant  had  approximately  35.07%  more  FPs  than  BCFTools  and  approximately  27.11%

more  FPs  than  GATK4.  This  comparison  underscores  VariantTransformer's  efficiency  in minimizing incorrect variant identifications-a crucial advantage in genomic analytics.

Furthermore,  the  comparison  with  DeepSVR 14 ,  a  recently  developed  deep  learning  model  to automate variant calling refinement, reveals significant distinctions 14 . DeepSVR classified SNPs into 'Pass', 'Fail', or 'Ambiguous' with Fliess' Kappa statistics 38 indicating fair agreement 14 . DeepSVR is designed and trained with a small training set (41,000 variants) specifically for deep coverage  (300-1000X)  cancer-type  samples,  processes  only  SNPs,  and  requires  extensive preprocessing of data 14 . In contrast, VariantTransformer supports both SNPs and InDels and does not require data preprocessing, allowing it to operate directly with VCF files from standard variant calling pipelines like BCFTools  and  GATK4  (a  detailed  comparison  of  DeepSVR  and VariantTransformer is presented in Supplementary Table S2). This adaptability makes VariantTransformer a more versatile tool suitable for a broader range of genomic studies.

However, there are limitations to our approach. The training dataset, derived mainly from wellcharacterized genomic regions provided by the GIAB consortium, might not fully represent the diversity of genomic scenarios encountered in wider research and clinical contexts. Moreover, the model configuration is primarily optimized for data generated from Illumina platforms. Future studies could expand the model's training scope to include more diverse and challenging genomic landscapes,  potentially  enhancing  its  applicability  and  accuracy  in  a  wider  array  of  genomic settings.

## Conclusion

VariantTransformer  represents  a  significant  advancement  in  the  field  of  genomic  studies, particularly in the refinement variant calling. Achieving an accuracy of 89.26%, this Transformerbased model surpasses traditional methods and rivals existing AI-based tools in performance. By conceptualizing  variant  calling  as  a  text  classification  problem,  VariantTransformer  efficiently differentiates between true variants and technical artifacts showcasing the robust application of NLP techniques in genomics. The demonstrated capabilities of VariantTransformer suggest its potential  to  substitute  conventional  threshold-based  filtering  of  low-coverage  genomic  data, offering a more accurate and efficient alternative. While the results are promising, the scope for further  enhancements  remains  vast.  Overall,  this  study  highlights  the  feasibility  of  using Transformer models within the realm of genomics and underscores the transformative potential of AI technologies in enhancing the accuracy and efficiency of variant calling, particularly in lowcoverage genomic studies, thus paving the way for significant advancements in the field.

## Acknowledgments

The authors wish to thank Génome Québec, Genome Canada, the government of Canada, the Ministère  de  l'Économie  et  de  l'Innovation  du  Québec,  the  Canadian  Field  Crop  Research Alliance, Semences Prograin Inc., Sollio Agriculture, Grain Farmers of Ontario, Barley Council of Canada, and Université Laval. The authors wish also to thank GIAB for providing the reference variants of the samples.

## Funding

This  work  was  funded  by  Genome  Canada  [#6548]  under  Genomic Applications  Partnership Program (GAPP).

## Competing Interests

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

## Authors' Contributions

Conceptualization, O.A. and D.T.; data curation, model development and formal analysis, O.A.; resources, D.T.; writing, review  and  editing, O.A., and  D.T.; supervision, D.T.; project administration, D.T.; funding acquisition, D.T. All authors have read and agreed to the published version of the manuscript.

## References

- 1. Syvänen, A. C. Accessing genetic variation: Genotyping single nucleotide polymorphisms. Nature Reviews Genetics vol. 2 930-942 Preprint at https://doi.org/10.1038/35103535 (2001).
- 2. Green, E. D. &amp; Guyer, M. S. Charting a course for genomic medicine from base pairs to bedside. Nature 2011 470:7333 470 , 204-213 (2011).
- 3. Van der Auwera, G. A. et al. From FastQ Data to High-Confidence Variant Calls: The Genome Analysis Toolkit Best Practices Pipeline. Curr Protoc Bioinformatics 43 , 11.10.1-11.10.33 (2013).
- 4. Hemstrom, W., Grummer, J. A., Luikart, G. &amp; Christie, M. R. Next-generation data filtering in the genomics era. Nature Reviews Genetics 2024 1-18 (2024) doi:10.1038/s41576-024-00738-6.
- 5. Carson, A. R. et al. Effective filtering strategies to improve data quality from population-based whole exome sequencing studies. BMC Nephrol 15 , 125 (2014).
- 6. Robinson, J. T. et al. Integrative genomics viewer. Nature Biotechnology vol. 29 24-26 Preprint at https://doi.org/10.1038/nbt.1754 (2011).
- 7. Robinson, J. T., Thorvaldsdóttir, H., Wenger, A. M., Zehir, A. &amp; Mesirov, J. P. Variant review with the integrative genomics viewer. Cancer Research vol. 77 e31-e34 Preprint at https://doi.org/10.1158/0008-5472.CAN-17-0337 (2017).
- 8. Lefouili, M. &amp; Nam, K. The evaluation of Bcftools mpileup and GATK HaplotypeCaller for variant calling in non-human species. Scientific Reports 2022 12:1 12 , 1-8 (2022).
- 9. Pfeifer, S. P . From next-generation resequencing reads to a high-quality variant data set. Heredity 2017 118:2 118 , 111-124 (2016).
- 10. Li, H. &amp; Durbin, R. Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics 25 , 1754-1760 (2009).
- 11. Langmead, B. &amp; Salzberg, S. L. Fast gapped-read alignment with Bowtie 2. Nature Methods 2012 9:4 9 , 357-359 (2012).
- 12. Spinella, J. F. et al. SNooPer: A machine learning-based method for somatic variant identification from low-pass next-generation sequencing. BMC Genomics 17 , 912 (2016).
- 13. Ding, J. et al. Feature-based classifiers for somatic mutation detection in tumour-normal paired sequencing data. Bioinformatics 28 , 167-175 (2012).

- 14. Ainscough, B. J. et al. A deep learning approach to automate refinement of somatic variant calling from cancer sequencing data. Nature Genetics 2018 50:12 50 , 1735-1743 (2018).
- 15. Strom, S. P. Current practices and guidelines for clinical next-generation sequencing oncology testing. Cancer Biol Med 13 , 3-11 (2016).
- 16. Vaswani, A. et al. Attention is All you Need. Adv Neural Inf Process Syst 30 , (2017).
- 17. Danecek, P. et al. Twelve years of SAMtools and BCFtools. Gigascience 10 , 1-4 (2021).
- 18. McKenna, A. et al. The genome analysis toolkit: A MapReduce framework for analyzing nextgeneration DNA sequencing data. Genome Res 20 , 1297-1303 (2010).
- 19. Zook, J. M. et al. Extensive sequencing of seven human genomes to characterize benchmark reference materials. Sci Data 3 , 1-26 (2016).
- 20. Li, H. et al. The Sequence Alignment/Map format and SAMtools. Bioinformatics 25 , 2078-2079 (2009).
- 21. Freed, D., Aldana, R., Weber, J. A. &amp; Edwards, J. S. The Sentieon Genomics Tools - A fast and accurate solution to variant calling from next-generation sequence data. bioRxiv 115717 (2017) doi:10.1101/115717.
- 22. Wagner, J. et al. Benchmarking challenging small variants with linked and long reads. Cell Genomics 2 , 100128 (2022).
- 23. Wolf, T., Debut, L., Sanh, V., Chaumond, J. &amp; ... HuggingFace's Transformers: State-of-the-art Natural Language Processing. ArXiv 1910.03771 , 1-8 (2019).
- 24. Loshchilov, I. &amp; Hutter, F. Decoupled Weight Decay Regularization. 7th International Conference on Learning Representations, ICLR 2019 (2017).
- 25. Hand, D. J. &amp; Till, R. J. A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems. Mach Learn 45 , 171-186 (2001).
- 26. Gorodkin, J. Comparing two K-category assignments by a K-category correlation coefficient. Comput Biol Chem 28 , 367-374 (2004).
- 27. Baldi, P ., Brunak, S., Chauvin, Y., Andersen, C. A. F. &amp; Nielsen, H. Assessing the accuracy of prediction algorithms for  classification: an overview. Bioinformatics 16 , 412-424 (2000).
- 28. Chicco, D. &amp; Jurman, G. The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics 21 , 1-13 (2020).
- 29. Jurman, G., Riccadonna, S. &amp; Furlanello, C. A Comparison of MCC and CEN Error Measures in Multi-Class Prediction. PLoS One 7 , e41882 (2012).
- 30. Godbole, S. &amp; Sarawagi, S. Discriminative Methods for Multi-labeled Classification. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 3056 , 22-30 (2004).

- 31. Varoquaux, G. et al. Scikit-learn. GetMobile: Mobile Computing and Communications 19 , 29-33 (2015).
- 32. Poplin, R. et al. A universal SNP and small-indel variant caller using deep neural networks. Nature Biotechnology 2018 36:10 36 , 983-987 (2018).
- 33. Hunter, J. D. Matplotlib: A 2D Graphics Environment. Comput Sci Eng 9 , 90-95 (2007).
- 34. Wickham, H. Elegant Graphics for Data Analysis: Ggplot2 . Applied Spatial Data Analysis with R (2008).
- 35. Krusche, P. et al. Best practices for benchmarking germline small-variant calls in human genomes. Nature Biotechnology 2019 37:5 37 , 555-560 (2019).
- 36. Brown, T. B. et al. Language Models are Few-Shot Learners. Adv Neural Inf Process Syst 2020December , (2020).
- 37. Fawcett, T. An introduction to ROC analysis. Pattern Recognit Lett 27 , 861-874 (2006).
- 38. McHugh, M. L. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 22 , 276 (2012).

Supplementary Table S1 . Training parameters employed during model development and final parameters used in the fine-tuned model. Default refers to pre-defined model parameters from Hugging Face documentation, while Model represents tuned parameters.

| Parameters   | Default          | VariantTransformer      |
|--------------|------------------|-------------------------|
| 30,522       | 30,567           | Vocab_size              |
| False        | VCF unique words | Never_split             |
| 768          | 256              | Hidden_size             |
| 12           | 8                | num_hidden_layers       |
| 12           | 8                | num_attention_heads     |
| 3072         | 512              | intermediate_size       |
| 512          | 256              | max_position_embeddings |
| Optional     | (zero) 0.1       | classifier_dropout      |

Supplementary Table S2. Comparison between DeepSVR and VariantTransformer models.

| Model                 | DeepSVR                                         | VariantTransformer                                                  |
|-----------------------|-------------------------------------------------|---------------------------------------------------------------------|
| General functionality | Filters SNPs from specific cancer types samples | Filters SNPs and InDels from any raw VCF file with no preprocessing |
| Variant type          | SNPs only                                       | SNPs and InDels                                                     |
| Training size         | 41,000 variants                                 | 2,000,000 variants                                                  |
| Model's spectrum      | Limited cancer types                            | Any sample                                                          |

| Training data preparation      | Manual review of a small number of variants from cancer cases   | Benchmark GIAB truth sets                                                                                      |
|--------------------------------|-----------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------|
| Input data type                | Requires preprocessing to a specific format with +50 features   | No preprocessing: Any VCF file produced from a conventional variant calling pipeline (i.e. BCFTools and GATK4) |
| Testing data depth of coverage | Deep coverage of 312X up to 1,000X for some cases               | Low coverage of 10X to 13.6X                                                                                   |
Omar Abdelwahab Davoud Torkamaneh
Refinement of genetic variants needs attention
Variant calling refinement is crucial for distinguishing true genetic
variants from technical artifacts in high-throughput sequencing data. Manual
review is time-consuming while heuristic filtering often lacks optimal
solutions. Traditional variant calling methods often struggle with accuracy,
especially in regions of low read coverage, leading to false-positive or
false-negative calls. Here, we introduce VariantTransformer, a
Transformer-based deep learning model, designed to automate variant calling
refinement directly from VCF files in low-coverage data (10-15X).
VariantTransformer, trained on two million variants, including SNPs and short
InDels, from low-coverage sequencing data, achieved an accuracy of 89.26% and a
ROC AUC of 0.88. When integrated into conventional variant calling pipelines,
VariantTransformer outperformed traditional heuristic filters and approached
the performance of state-of-the-art AI-based variant callers like DeepVariant.
Comparative analysis demonstrated VariantTransformer's superiority in
functionality, variant type coverage, training size, and input data type.
VariantTransformer represents a significant advancement in variant calling
refinement for low-coverage genomic studies.