MAESTRO: Masked Encoding Set Transformer with Self-Distillation
Matthew Eric Lee Jaesik Kim Matei Ionita Jonghyun Lee Michelle L. McKeague YONGHYUN NAM Irene Khavin Yidi Huang Victoria Fang Sokratis Apostolidis Divij Mathew Shwetank Ajinkya Pattekar Zahabia Rangwala Amit Bar-Or Benjamin A Fensterheim Benjamin A. Abramoff Rennie L. Rhee Damian Maseda Allison R Greenplate et al. (2 additional authors not shown)
The interrogation of cellular states and interactions in immunology research is an ever-evolving task, requiring adaptation to the current levels of high dimensionality. Cytometry enables high-dimensional profiling of immune cells, but its analysis is hindered by the complexity and variability of the data. We present MAESTRO, a self-supervised set representation learning model that generates vector representations of set-structured data, which we apply to learn immune profiles from cytometry data. Unlike previous studies only learn cell-level representations, whereas MAESTRO uses all of a sample's cells to learn a set representation. MAESTRO leverages specialized attention mechanisms to handle sets of variable number of cells and ensure permutation invariance, coupled with an online tokenizer by self-distillation framework. We benchmarked our model against existing cytometry approaches and other existing machine learning methods that have never been applied in cytometry. Our model outperforms existing approaches in retrieving cell-type proportions and capturing clinically relevant features for downstream tasks such as disease diagnosis and immune cell profiling.
self-supervision, representation learning, immunology, biology, single-cell, cytometry, set, set representations
We developed a self-supervised set representation learning model for vector-sized representations of single-cell data