Molecular representation learning plays an essential role in cheminformatics. Recently, language model-based approaches have gained popularity as an alternative to traditional expert-designed features to encode molecules. However, these approaches only utilize a single molecular language for representation learning. Motivated by the fact that a given molecule can be described using different languages such as Simplified Molecular Line Entry System (SMILES), the International Union of Pure and Applied Chemistry (IUPAC), and the IUPAC International Chemical Identifier (InChI), we propose a multilingual molecular embedding generation approach called MM-Deacon ( m ultilingual m olecular d omain e mbedding a nalysis via con trastive learning). MM-Deacon is pre-trained using SMILES and IUPAC as two different languages on large-scale molecules. We evaluated the robustness of our method on seven molecular property prediction tasks from MoleculeNet benchmark, zero-shot cross-lingual retrieval, and a drug-drug interaction prediction task. Drug discovery process involves screening of millions of compounds in the early stages of drug design, which is time consuming and expensive. Computer-aided drug discovery can reduce the time and cost involved in this process via automating various cheminformatics tasks (Kontogeorgis and Gani, 2004; Xu et al., 2017; Winter et al., 2019). Traditional methods to encode molecules such as fingerprint generation rely heavily on molecular fragment-level operations on top of molecule graph constructed by molecular atoms and bonds (Burden, 1989; Bender and Glen, 2004; Vogt and Bajorath, 2008; Muegge and Mukherjee, 2016). An example of such methods is Morgan fingerprint, also known as Extended-Connectivity Fingerprint (ECFP) (Morgan, 1965; Rogers and Hahn, 2010), where a fixed binary hash function is applied on each atom and its neighborhood. These kinds of approaches focus on local features, hence they may not capture global information. In addition to molecule graph, a given molecule can also be described with different languages such as Simplified Molecular Line Entry System (SMILES), the International Union of Pure and Applied Chemistry (IUPAC), and the IUPAC International Chemical Identifier (InChI). Particularly, SMILES is widely used to represent molecule structures as ASCII strings (Weininger, 1988; Favre and Powell, 2013) at an atom and bond level. IUPAC nomenclature, on the other hand, serves the purpose of systematically naming organic compounds by basic words that indicate the structure of the compound and prioritize on functional groups to facilitate communication (Panico et al., 1993). Fig. 1 shows a comparison of SMILES and IUPAC characteristics for the same molecule. The SMILES string is created by traversing the molecule graph, where each letter in the SMILES string (such as C, F, N, O in Fig. 1) corresponds to an atom on the graph, and other characters represent positions and connectivity. However, IUPAC names are akin to a natural language, and morphemes in the IUPAC name (like fluoro, prop, en, yl in this example) often represent specific types of substructure on the molecule graph, which are also responsible for characteristic chemical reactions of molecules. Advances in natural language processing (NLP) have been very promising for molecule embedding generation and molecular property prediction (Xu et al., 2017; Gmez-Bombarelli et al., 2018; Samanta et al., 2020; Koge et al., 2021; Honda et al., 2019; Shrivastava and Kell, 2021; Goh et al., 2017; Schwaller et al., 2019; Payne et al., 2020; Aumentado-Armstrong, 2018). It is important to note that all of the methods mentioned above work with SMILES representation only. Therefore, the underlying chemical knowledge encoded in the em-3441 Figure 1: Schematic illustration of differences in SMILES and IUPAC characteristics for the same molecule. bedding is restricted to a single language modality. Transformer models trained with self-supervised masked language modeling (MLM) loss (Vaswani et al., 2017) in chemical domain (Wang et al., 2019; Chithrananda et al., 2020; Elnaggar et al., 2020; Rong et al., 2020; Schwaller et al., 2021; Bagal et al., 2021) have also been used for molecular representation learning. However, pre-training objectives like MLM loss tend to impose task-specific bias on the final layers of Transformers (Carlsson et al., 2020), limiting the generalization of the embeddings. In recent years, contrastive learning has been successful in multimodal vision and language research (Radford et al., 2021; Meyer et al., 2020; Shi et al., 2020; Cui et al., 2020; Chen et al., 2021; Alayrac et al., 2020; Akbari et al., 2021; Lee et al., 2020; Liu et al., 2020). Radford et al. (2021) used image-text pairs to learn scalable visual representations. Carlsson et al. (2020) showed the superiority of contrastive objectives in acquiring global (not fragment-level) semantic representations. In light of these advances, we propose MM-Deacon ( m ultilingual m olecular d omain e mbedding a nalysis via con trastive learning), a molecular representation learning algorithm built on SMILES and IUPAC joint training. Transformers are used as base encoders in MM-Deacon to encode SMILES and IUPAC, and embeddings from encoders are projected to a joint embedding space. Afterwards, a contrastive objective is used to push the embeddings of positive cross-lingual pairs (SMILES and IUPAC for the same molecule) closer together and the embeddings of negative cross-lingual pairs (SMILES and IUPAC for different molecules) farther apart. Here instead of using SMILES and IUPAC for sequence-to-sequence translation (Rajan et al., 2021; Krasnov et al., 2021; Handsel et al., 2021), we obtain positive and negative SMILES-IUPAC pairs and contrast their embeddings at the global molecule level rather than the fragment level. Different molecule descriptors are thus integrated into the same joint embedding space, with mutual information maximized across distinct molecule languages. We pre-train MM-Deacon on 10 million molecules chosen at random from the publicly available PubChem dataset (Kim et al., 2016) and then use the pre-trained model for downstream tasks. Our main contributions are as follows:  We propose MM-Deacon, a novel approach for utilizing multiple molecular languages to generate molecule embeddings via contrastive learning. To the best of our knowledge, we are the first to leverage mutual information shared across SMILES and IUPAC for molecule encoding. We conduct extensive experiments on a variety of tasks, including molecular property prediction, cross-lingual molecule retrieval, and drug-drug interaction (DDI) prediction, and demonstrate that our approach outperforms baseline methods and existing state-of-the-art approaches. Deep learning tasks commonly face two challenges: first, dataset size is often limited, and second, annotations are scarce and expensive. A pre-training scheme can benefit downstream tasks by leveraging large-scale unlabeled or weakly labeled data. Such pre-training and fine-tuning frameworks have recently sparked much interest in the molecular domain (Hu et al., 2019; Samanta et al., 2020; Chithrananda et al., 2020; Rong et al., 2020; Shrivastava and Kell, 2021; Xue et al., 2021; Zhu et al., 2021; Wang et al., 2021; Liu et al., 2021). Existing pre-training methods can be divided into three categories based on the models used: pre-training with graph neural networks (GNNs), pre-training with language models, and pre-training with hybrid models. nodes and bonds as edges. Hu et al. (2019) pretrained GNNs on 2 million molecules using both node-level and graph-level representations with attribute masking and structure prediction objectives. MolCLR (Wang et al., 2021) used subgraph-level molecule data augmentation scheme to create positive and negative pairs and contrastive learning to distinguish positive from negative. GraphMVP (Liu et al., 2021) was pre-trained on the consistency of 2D and 3D molecule graphs (3D graphs formed by adding atom spatial positions to 2D graphs) and contrastive objectives with GNNs. Pre-training with language models. Language models are widely used to encode SMILES for molecular representation learning. Xu et al. (2017) reconstructed SMILES using encoder-decoder gated recurrent units (GRUs) with seq2seq loss, where embeddings in the latent space were used for downstream molecular property prediction. Chemberta (Chithrananda et al., 2020) fed SMILES into Transformers, which were then optimized by MLM loss. FragNet (Shrivastava and Kell, 2021) used encoder-decoder Transformers to reconstruct SMILES and enforced extra supervision to the latent space with augmented SMILES and contrastive learning. X-Mol (Xue et al., 2021) was pretrained by taking as input a pair of SMILES variants for the same molecule and generating one of the two input SMILES as output with Transformers on 1.1 billion molecules. Pre-training with hybrid models. Different molecule data formats can be used collaboratively to enforce cross-modality alignment, resulting in the use of hybrid models. For example, DMP (Zhu et al., 2021) was built on the consistency of SMILES and 2D molecule graphs, with SMILES encoded by Transformers and 2D molecule graphs encoded by GNNs. Unlike other molecule pre-training methods, MM-Deacon is multilingually pre-trained with language models using pairwise SMILES and IUPAC. Compared with using molecule graphs with GNNs, IUPAC names encoded by language models bring in a rich amount of prior knowledge by basic words representing functional groups, without the need for sophisticated graph hyperparameter design. MM-Deacon is a deep neural network designed for SMILES-IUPAC joint learning with the goal of contrasting positive SMILES-IUPAC pairs from nega-Figure nega-Figure 2: Schematic diagram for MM-Deacon pretraining. SMILES and IUPAC are encoded by separate Transformers. Embeddings from encoders are average-pooled globally and projected to a joint chemical embedding space, where contrastive objectives are used to maximize mutual information for SMILES and IUPAC from the same molecule and distinguish SMILES and IUPAC from different molecules. tive pairs and thus maximizing mutual information across different molecule languages. SMILES and IUPAC for the same molecule are regarded as positive pairs, while SMILES and IUPAC for different molecules are considered negative. Transformer encoders with multi-head self-attention layers are utilized to encode SMILES and IUPAC strings. Embeddings from the encoders are pooled globally and projected to the joint chemical embedding space. MM-Deacon is pre-trained on a dataset of 10 million molecules chosen at random from PubChem. We use a Byte-Pair Encoding (BPE) tokenizer for SMILES tokenization, as is shown by Chithrananda et al. (2020) that BPE performed better than regex-based tokenization for SMILES on downstream tasks. For IUPAC name tokenization, a rule-based regex (Krasnov et al., 2021) that splits IUPAC strings based on suffixes, prefixes, trivial names, and so on is employed. The input sequence length statistics as well as the top 20 most frequent tokens in the SMILES and IUPAC corpora are displayed in Figs. 9 and 10 (Appendix A). As illustrated in Fig. 2, MM-Deacon takes SMILES and IUPAC strings as the input to separate branches. The input text string s is tokenized and embedded into a numeric matrix representation x within each branch, and the order of the token list is preserved by a positional embedding p x . Then x and p x are ingested by an encoder block  that consists of 6 layers of Transformer encoder. A Trans-3443 former encoder has two sub-layers, a multi-head attention layer and a fully-connected feed-forward layer. Each sub-layer is followed by a residual connection and layer normalization to normalize input values for all neurons in the same layer (Vaswani et al., 2017; Ba et al., 2016). The multi-head attention layer acquires long-dependency information by taking all positions into consideration. We then use a global average pooling layer  to integrate features at all positions and a projection layer  to project the integrated feature vector to the joint embedding space. Thus the final embedding z of x can be expressed as, z ( x ) =  (  (  ( x + p x ))) . The maximum input token sequence length is set to 512. For each of the 6 Transformer encoder layers, we choose the number of self-attention heads as 12 and hidden size of 768. The projection layer  projects the vector from length of 768 to 512 to make the representation more compact. Thus z ( x )  R 512 . Our goal is to align pairs of language modalities in the joint embedding space by maximizing mutual information of positive pairs and distinguishing them from negative pairs. For this purpose, we use InfoNCE (Oord et al., 2018; Alayrac et al., 2020; Radford et al., 2021) as the contrastive loss. We do not construct negative pairs manually. Instead, during training, we obtain negative pairs in mini-batches. Using a minibatch of NSMILES-IUPAC pairs from N molecules as input, N positive pairs and N 2  N negative pairs can be generated within the correlation matrix of NSMILES strings and N IUPAC strings. More specifically, the only positive pair for i -th SMILES is i -th IUPAC, while the remaining N  1 IUPAC strings form negative pairs with i -th SMILES. Therefore, the InfoNCE loss for i -th SMILES is, L sli =  log ( exp ( sim ( z sli , z ipi ) / ) (cid:80) Nj =1 exp ( sim ( z sli , z ipj ) / )) , (2) where sl and ip represent SMILES and IUPAC respectively. sim () is the pairwise similarity function that employs cosine similarity in this work. is the temperature. Likewise, the loss function for i -th IUPAC is, Figure 3: Possible scenarios in the downstream stage. L ip i =  log ( exp ( sim ( z sli , z ipi ) / ) (cid:80) Nj =1 exp ( sim ( z sl j , z ip i ) / )) . (3) As a result, the final loss function is as follows, L = 1 2 N (cid:88) t { sl,ip } N (cid:88) i =1 L ti . (4) We pre-train MM-Deacon on 80 V100 GPUs for 10 epochs (15 hours in total) with a 16 batch size on each GPU using AdamW optimizer with a learning rate of 10  6 . The temperature  is set as 0.07 as in (Oord et al., 2018). Knowledge gained during pre-training can be transferred to downstream tasks in different ways. Fig. 3 lists two situations that make use of pre-trained MM-Deacon in the downstream stage. MM-Deacon fine-tuning: A task-specific clas-sification/regression head can be attached to pretrained MM-Deacon and the system as a whole can be tuned on downstream task datasets. MM-Deacon fingerprint: Pre-trained MM-Deacon is frozen. An input molecule is embedded as MM-Deacon fingerprint for zero-shot explorations (such as clustering analysis and similarity retrieval) and supervised tasks with the help of an extra classifier. MM-Deacon was evaluated on seven molecular property prediction tasks from MoleculeNet bench-3444 mark (Wu et al., 2018), zero-shot cross-lingual retrieval, and a drug-drug interaction (DDI) prediction task. MoleculeNet benchmark provides a unified framework for evaluating and comparing molecular machine learning methods on a variety of molecular property prediction tasks ranging from molecular quantum mechanics to physiological themes, and is widely acknowledged as the standard in the research community (Hu et al., 2019; Chithrananda et al., 2020; Xue et al., 2021; Zhu et al., 2021; Wang et al., 2021; Liu et al., 2021). Four classification datasets and three regression datasets from the MoleculeNet benchmark were utilized to evaluate our approach. Data. The blood-brain barrier penetration (BBBP), clinical trail toxicity (ClinTox), HIV replication inhibition (HIV), and side effect resource (SIDER) datasets are classification tasks in which molecule SMILES strings and their binary labels are provided in each task. Area Under Curve of the Receiver Operating Characteristic curve (ROC-AUC) is the performance metric in which the higher the value, the better the performance. For datasets with multiple tasks like SIDER, the averaged ROC-AUC across all tasks under the same dataset is reported. The fractions of train/val/test sets for each classification task are 0.8/0.1/0.1 with Scaffold split. Note that data split using molecule scaffolds (two-dimensional structural frameworks) results in more structurally distinct train/val/test sets, making it more challenging than random split (Wu et al., 2018). The water solubility data (ESOL), free solvation (FreeSolv), and experimental results of octabol/water distribution coefficient (Lipophilic-ity) datasets are all regression tasks to predict numeric labels given molecule SMILES strings. Root Mean Square Error (RMSE) is used as the evaluation metric in which the lower the value, the better the performance. As recommended by MoleculeNet, random split that divides each dataset into 0.8/0.1/0.1 for train/val/test sets is employed. The results on validation set are used to select the best model. To maintain consistency with MoleculeNet, we ran each task three times, each time with a different data split seed, to obtain the mean and standard deviation (std) of the metric. Details of each dataset such as the number of tasks and molecules it contains are displayed in Table 1. Model. We utilized the model shown in Fig. 3(a) in which a linear layer serving as the task-specific head was added to pre-trained MM-Deacon SMILES branch for fine-tuning (IUPAC branch was removed). Cross-entropy loss was employed for classification tasks and MSE loss was employed for regression tasks. Hyperparameter tuning was performed using grid search with possible choices listed in Table 5 (Appendix B). Each task was optimized individually. Results. Table 2 shows the mean and std results for each dataset. The first half of the table displays results imported from MoleculeNet (Wu et al., 2018), while the second section shows the results from MM-Deacon and other state-of-the-art molecular pre-training and fine-tuning approaches. MLM[CLS] denotes our implementation of a Chemberta (Chithrananda et al., 2020) variant that uses the same Transformer settings as MM-Deacon SMILES branch, pre-trained with MLM loss on 10M molecules, and fine-tuned through [CLS] token with the same downstream setting as MM-Deacon. MM-Deacon exceeds the performance of traditional machine learning methods like random forest (RF) and task-specific GNNs reported in MoleculeNet work by a significant margin for most of the tasks. When compared to other pre-training based approaches, MM-Deacon outperforms the existing state-of-the-art approaches in four of the seven datasets and is comparable in the remaining three, with major improvements on ClinTox and FreeSolv. All pre-training based methods were pre-trained on millions of molecules, with the exception of GraphMVP, which was pre-trained on 50K molecules. The requirement that molecules have both 2D and 3D structure information available at 3445 Method BBBP ClinTox HIV SIDER ESOL FreeSolv Lipophilicity RF 71.4  0.0 71.3  5.6 78.1  0.6 68.4  0.9 1.07  0.19 2.03  0.22 0.876  0.040 KernelSVM 72.9  0.0 66.9  9.2 79.2  0.0 68.2  1.3 -Multitask 68.8  0.5 77.8  5.5 69.8  3.7 66.6  2.6 1.12  0.15 1.87  0.07 0.859  0.013 GC 69.0  0.9 80.7  4.7 76.3  1.6 63.8  1.2 0.97  0.01 1.40  0.16 0.655  0.036 Weave 67.1  1.4 83.2  3.7 70.3  3.9 58.1  2.7 0.61  0.07 1.22  0.28 0.715  0.035 MPNN --0.58  0.03 1.15  0.12 0.719  0.031 Hu et al. (2019) 70.8  1.5 78.9  2.4 80.2  0.9 65.2  0.9 -MolCLR (Wang et al., 2021) 73.6  0.5 93.2  1.7 80.6  1.1 68.0  1.1 -DMP (Zhu et al., 2021) 78.1  0.5 95.0  0.5 81.0  0.7 69.2  0.7 --X-Mol (Xue et al., 2021) 96.2  N/A 98.4  N/A 79.8  N/A -0.578  N/A 1.108  N/A 0.596  N/A GraphMVP (Liu et al., 2021) 72.4  1.6 77.5  4.2 77.0  1.2 63.9  1.2 1.029  N/A -0.681  N/A MLM[CLS] 70.6  4.5 93.2  0.1 77.9  0.2 64.8  1.3 0.640  0.023 1.21  0.046 0.804  0.037 MM-Deacon 78.5  0.4 99.5  0.3 80.1  0.5 69.3  0.5 0.565  0.014 0.926  0.013 0.650  0.021 Table 2: Results in terms of mean and std for each dataset included from MoleculeNet benchmark. the same time to be qualified has limited the scala-bility of GraphMVP. MM-Deacon and MLM-CLS both used 6 layers of Transformer blocks to process SMILES. For each task, MM-Deacon, which was pre-trained with both SMILES and IUPAC, outscored MLM-CLS , which was pre-trained with SMILES only. MM-Deacon and DMP performed comparably on the four classification tasks, while DMP used 12 layers of Transformer blocks for SMILES and a 12-layer GNN to encode a molecule 2D graph, which is nearly twice the size of MM-Deacon model. Moreover, we found that BBBP test set is significantly more challenging than the validation set, which is consistent with the results published in the MoleculeNet paper (Wu et al., 2018). The substantially high accuracy X-Mol achieved on the BBBP dataset could be due to either the 1.1 billion molecules they utilized for pre-training or a different dataset division approach they employed. In addition to conducting fine-tuning on supervised tasks like molecular property prediction, pretrained MM-Deacon can be employed directly in large-scale zero-shot analysis. Zero-shot cross-lingual retrieval operates on top of MM-Deacon fingerprint generated by pre-trained MM-Deacon given molecule SMILES or IUPAC as input. This task enables the retrieval of similar molecules across languages without the need for translation, and it can also be used to evaluate the learned agreement in the joint embedding space between SMILES and IUPAC representations. Data. 100K molecules were randomly chosen from PubChem dataset after excluding the 10 million molecules used for MM-Deacon pre-training. SMILES and IUPAC strings are provided for each molecule. We used average recall at K (R@1 and R@5) to measure the percentage of the ground truth that appears in the top K retrieved molecules. Model. Pre-trained MM-Deacon was used for MM-Deacon fingerprint generation, as shown in Fig. 3(b). As a result, each SMILES and IUPAC string was encoded as MM-Deacon SMILES fingerprint and IUPAC fingerprint respectively. Cosine similarity between a query and molecules in the search candidates was used to determine the ranking. Results. Fig. 4 shows the outcomes of SMILES-to-IUPAC and IUPAC-to-SMILES retrieval in terms of recall. We not only performed retrieval directly on the entire 100K molecules, but also reported the results on smaller groups of molecules (100, 10K) to get a more thorough picture of the retrieval performance. MM-Deacon gets a R@5 above 85% for both types of cross-lingual retrieval even while executing retrieval on 100K molecules. Moreover, Figs. 5 and 6 show an example of SMILES-to-IUPAC retrieval and an example of IUPAC-to-SMILES retrieval respectively. Additional retrieval examples for scenarios where the performance is difficult to be quantified, such as retrieval queried by a free combination of 3446 Figure 4: Average recall for cross-lingual retrieval on groups of molecules with different sizes. The effectiveness of combining MM-Deacon fingerprints with a task-specific classifier for supervised learning was tested on a DDI prediction task. The objective of this task is to predict whether or not Table 3: DDI prediction results using 5-fold cross-validation. Data. The DDI dataset (Zhang et al., 2017) used here includes 548 drugs, with 48,584 known interactions, and 101,294 non-interactions (may contain undiscovered interactions at the time the dataset was created). We obtained the SMILES and IUPAC names for each drug from PubChem. Stratified 5-fold cross-validation with drug combination split was utilized. The evaluation metrics are Area Under the ROC Curve (AUC), Area Under the Precision-Recall Curve (AUPR), precision, and recall, with AUPR serving as the primary metric (Zhang et al., 2017). Model. MM-Deacon fingerprints of paired drugs are concatenated and fed into a multi-layer percep-tron (MLP) network implemented by scikit-learn (Pedregosa et al., 2011) for binary classification. Three different types of fingerprints are used for MM-Deacon: SMILES, IUPAC, and concatenated SMILES and IUPAC fingerprints. The MLP has one hidden layer with 200 neurons. ReLU activation and a learning rate of 10  3 are used. Results. As shown in Table 3, MM-Deacon outperforms other methods in terms of AUPR, precision and recall, with the maximum AUPR obtained when SMILES and IUPAC fingerprints were concatenated as input feature set. Ensemble models (Zhang et al., 2017) included extra bioactivity related features in addition to drug structural properties. DPDDI (Feng et al., 2020) encoded molecule graph with GNNs, from which latent features were concatenated for pairs of drugs and ingested into a deep neural network. Table 4 shows the top 20 most potential interactions predicted by MM-Deacon (concat) in the non-interaction set (false positives), 13 out of which are confirmed as true positives by DrugBank 1 . While, the number is 7/20 for ensemble models (Zhang et al., 2017). After being pre-trained on 10 million molecules, MM-Deacon showed outstanding knowledge transfer capabilities to various downstream scenarios (Fig. 3) where a pre-trained model could be used. The competitive performance on seven molecular property prediction tasks from MoleculeNet benchmark demonstrated the effectiveness of the pre-trained MM-Deacon when adopting a network fine-tuning scheme as shown in Fig. 3(a). The evaluation results of zero-shot cross-lingual retrieval further revealed that MM-Deacon SMILES and IUPAC fingerprints shared a substantial amount of mutual information, implying that an IUPAC name can be used directly without first being translated to SMILES format as chemists have done in the past. The DDI prediction task showed that MM-Deacon also allows directly using embeddings in the joint cross-modal space as molecular fingerprints for downstream prediction tasks, which is a widely used strategy in cheminformatics. MM-Deacon profited from the alignment of two molecule languages with distinct forms of nomenclatures, as opposed to the baseline MLM[CLS] model, which was pre-trained on SMILES representation only. Furthermore, we looked at molecule-level and token-level alignments of MM-Deacon to untangle the outcome of cross-lingual contrastive learning. We used centered kernel alignment (CKA) (Ko-rnblith et al., 2019) with RBF kernel to compare representations between different layers. In Fig. 7(a), the representations of 6 Transformer layers and the final projection layer were compared between MM-Deacon SMILES and IUPAC branches, where the representations differ in shallow layers, while reach a high level of alignment in deeper layers. In Fig. 7(b), both the MM-Deacon SMILES branch and MLM[CLS] model take SMILES as the input, therefore the shallow layers have a high alignment score, while the representation varies as the network grows deeper. Fig. 7 shows that MM-Deacon aligned SMILES and IUPAC representations effectively, and that molecular representations trained with SMILES and IUPAC differs from representations trained only on SMILES. The cosine similarity matrix of MM-Deacon fingerprints between tokens from the IUPAC corpus and tokens from the SMILES corpus is shown in Fig. 8. The table in Fig. 8 lists IUPAC tokens expressed in SMILES language, and the heat map demonstrates that there exists a good token-level alignment between SMILES and IUPAC. In this study, we proposed a novel method for multilingual molecular representation learning that combines mutual information from SMILES-IUPAC joint training with a self-supervised contrastive loss. We evaluated our approach for molecular property prediction, zero-shot cross-lingual retrieval, and DDI prediction. Our results demonstrate that the self-supervised multilingual contrastive learning framework holds enormous possibilities for chemical domain exploration and drug discovery. In future work, we plan to scale MM-Deacon pretraining to larger dataset sizes, as well as investigate the applicability of MM-Deacon to other types of molecule languages. We would like to thank Min Xiao and Brandon Smock for some insightful discussions.
abstain abstain abstain objective abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain other abstain abstain objective abstain abstain abstain abstain method objective objective objective abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain method method method method abstain abstain abstain method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain objective method objective objective other