Raman Spectrum Matching with Contrastive Representation Learning

Raman spectroscopy is an effective, low-cost, non-intrusive technique often used for chemical identification. Typical approaches are based on matching observations to a reference database, which requires careful preprocessing, or supervised machine learning, which requires a fairly large number of training observations from each class. We propose a new machine learning technique for Raman spectrum matching, based on contrastive representation learning, that requires no preprocessing and works with as little as a single reference spectrum from each class. On three datasets we demonstrate that our approach significantly improves or is on par with the state of the art in prediction accuracy, and we show how to compute conformal prediction sets with specified frequentist coverage. Based on our findings, we believe contrastive representation learning is a promising alternative to existing methods for Raman spectrum matching.


Introduction
tion. 4 Raman spectroscopy is often used in these applications 2-4 as a low-cost, non-invasive technique. Raman spectroscopy provides a chemical fingerprint by using scattered light to probe the vibrational modes of a sample. 2 When photons from a laser interact with the molecule, the scattered photons are shifted in frequency, and a spectrum is measured containing peaks representative of the chemical composition of the substance. 5 Substance identification is then usually carried out by matching a measured spectrum to a reference library. This process can be automated by computing a similarity score between the acquired spectrum and each spectra in the library. 6 The computation of the similarity scores usually involves preprocessing the spectra in terms of baseline correction, smoothing, and normalization. 7,8 Matrix factorization methods such as multivariate curve resolution with non-negativity constraints (also known as non-negative matrix factorization) are often applied to extract features from the spectra. [9][10][11] These features are then commonly used as input to machine learning classifiers such as support vector machines (SVM), random forest (RF), 12 K nearest neighbor (KNN) 2 classifiers, or tree-based method. 13,14 Appropriate preprocessing is essential for these methods to work well in practice, and their design requires extensive domain knowledge. 7 Handling the preprocessing and model training as separate steps may result in inadvertently removing useful spectral information. 5 Additionally, these approaches may fail when dealing with complex spectra. 3,7,15 To alleviate this problem, we investigate how deep neural networks can be used to jointly learn a feature representation and a classifier by optimizing the entire processing end-to-end.
Deep neural networks have achieved superior performance across many fields. 2,[16][17][18] In the context of spectral matching, neural networks are usually trained in a supervised manner, optimized directly to map an observed spectrum to a score for each reference substance in the training database. 4,5,7,19 Typically a convolutional neural network (CNN) is used since it can detect similarities in peaks and patterns across different wavenumbers, 20 leading to better sample efficiency. For example, Ho et al. 4 identify bacteria with high accuracy on low signal-to-noise spectra using a ResNet 17 -based classifier. However, the performance of CNN-based supervised classifiers strongly depends on the amount of training data, and in many practical cases only a few spectra are available per substance. 5 Additionally, if the reference database is modified, a supervised classifier needs to be retrained, which induces impractical computational costs. 5 In use-cases such as detection of trace amounts in chemical threat monitoring, where responders need to decide on further actions, it is important that the classification uncertainty is appropriately characterized. When the output of a spectrum matching algorithm is given as a probability of detection, the score must be well-calibrated. It is well known that deep neural networks do not inherently output well-calibrated probabilities. 21 To ensure that the prediction scores are well-calibrated, further statistical methods need to be utilized. Reporting a set of candidate detections can instead be useful in cases where it is critical to discover the presence of a substance. In this way, the responders can screen for the existence of a target substance at a certain probability threshold.
In this paper, we present a method for precise, data-efficient, and uncertainty-calibrated classification of Raman spectra. Rather than training a supervised model to classify a spectrum among a set of classes, we train a neural network to predict if two spectra belong to the same class or not. This is done by processing two spectra by the same Siamese network, and computing a similarity score between their output feature representations. After training the network, a new spectrum can be classified by computing its representation and similarity score with reference spectra. This approach makes it easy to understand how the model works, as it supplies both a class label and the library spectra used in the match. A further benefit, not explored in this work, is that the reference database can be modified without retraining the model. Finally, we propose a conformal 22,23 procedure to compute a prediction set that is guaranteed to include the true class with a high user-specified probability. This is demonstrated on a Surface Enhanced Raman Spectroscopy (SERS) dataset, where a large number of spectra are available per class.
Our proposed system includes several technical novelties, including • an optimized neural network architecture for neural feature representation learning using depth-wise separable convolutions, • a novel procedure for data augmentation that improves learning with even a single spectrum from each class, • a tailored conformal prediction set procedure with a frequentist guarantee of including the correct class prediction.
We experimentally validate our proposed approach on three publicly available datasets, and demonstrate that we match or outperform the existing state-of-the-art. Our model does not rely on data preprocessing, and in fact performs significantly better on raw data. Finally, we demonstrate that our conformal prediction sets in practice include the correct class label on held-out test data with approximately the prescribed frequency and can be tuned to provide near-perfect classification sets of moderate size.

Sample description
We demonstrate our system on three publicly available datasets (two Raman and one SERS) which we refer to as Mineral, Organic, and Bacteria, summarized below and in Table 1.
Mineral: Minerals from the RRUFF 6 database. The spectra from each mineral are recorded under different measuring environments and with different wavelengths, and we use both the raw (r) preprocessed (p) spectra.
Organic: Organic compounds collected and measured with several different excitation sources as described by Zhang et al. 24 . We use both the raw (r) and preprocessed (p) spectra.
Bacteria: Bacterial Raman spectra described by Ho et al. 4 . This dataset consists of: reference dataset, reference-finetune dataset, and test dataset. The test dataset is

Model architecture
We propose a contrastive representation learning method based on a Siamese neural network. 25 In this approach, the multiclass classification problem is re-cast as binary-classification by organizing the spectra into pairs and predicting whether the paired spectra are from the same or different classes, referred to as a positive or negative pair. This classification is achieved by first mapping the two spectra into learned feature representations using an identical representation network for each spectrum. Next, a second neural network is used to compute a similarity measure in the feature domain, resulting in a similarity score.

Representation network
The goal of the representation network is to extract spectral features that are useful for   number of parameters in the neural network will increase drastically, and while this makes the model more expressive it also demands more training data.
As a compromise between expressivity and data efficiency, we propose to incorporate Xception, 16,26,27 also known as depthwise seperable convolutions. This allows the neural net learn richer and more representative features while keeping the number of parameters low. Xception assumes that the the cross-channel and cross-wavenumber correlations are decoupled and thus can be mapped separately: The convolutional operation is factorized into a point-wise convolution along the feature channel dimension followed by a channel-wise convolution along the wavenumber dimension.
The architecture of our model can be seen in Fig. 2. Since depthwise separable convolutions are most effective in high dimensions, 16,26 we first use two conventional convolutional blocks to transform the one-dimensional input into a larger number of channels. Each convolutional block contains two times of one-dimensional convolutional layers, batch normalization layers, and LeakyReLu layers. We use only a single pooling layer along the wavenumber dimension to void losing accurate peak location information. We then stack another two Xception blocks on top as shown in Fig. 2. We use residual connections 17 in the Xception block since they have been shown to be essential in helping with convergence in terms of both speed and accuracy. 16,17,26 Each block ends with a dropout layer to alleviate the risk of over-fitting the model. The produced feature maps from the last Xception block are used as the final representation.

Similarity network
To quantify the similarity of two spectra, we first calculate the element-wise product and absolute difference between their feature maps F 1 and F 2 , where c and w are the channel and wavenumber feature index. This results in two distance maps D prod. and D diff. . Akin to dot product/cosine similarity, D prod. will be large when two spectra have strong features in common at the same wavenumber, whereas D diff. will be small where the two spectra have similar features irrespective of their magnitude. We concatenate the two distance maps along the wavenumber dimension and pass the resulting matrix to a 1D convolutional layer (kernel size 1, stride 1) and a fully connected layer followed by a sigmoid activation to yield the final similarity score p ∈ (0, 1). In Fig. 3 we illustrate our framework using a pair of example spectra as input.

Similarity score matching
At inference time, we first calculate the similarity score between a test spectrum and each of the spectra in the reference library. We then group these similarity scores based on the reference label and take the maximum similarity score within each group. For the RRUFF  Figure 3: The proposed spectra matching approach. During training, we first extract representative features from the input spectra using a deep neural network. We then quantify the similarity by calculating the distance between the extracted feature maps using both element-wise multiplication (D prod. ) and absolute element-wise difference (D diff. ). The values in the bracket in the x-axis correspond to the wavenumber index in the input spectrum. These two distance maps are then concatenated along the wavenumber dimension and used to produce a single similarity score p ∈ (0, 1). After training, all spectra are allocated to the reference library, and at inference time, we calculate the similarity score between the unknown spectrum and each reference match by the similarity score.
is experimentally optimized using the validation dataset.

Model training
Given a paired set of input spectra and the corresponding labels, we train the model using a cross-entropy objective with an L 2 regularization term where N is the number of pairs per batch, y i is the label for the pair i (y i = 1 for a positive pair and y i = 0 for a negative pair), and p i is the similarity score for pair i. The L 2 regularization term is included to alleviate over-fitting, with λ = 10 −3 chosen experimentally. We train the model with Adam Optimizer with the initial learning of 5 · 10 −5 , which is decayed following cosine annealing schedule. 28 During training, the dropout rate in each dropout layer is set to be 0.5, 7 and we deactivate dropout layers during testing. Once the training is done, we allocate all the learned representations from the training data to the reference library.

Data augmentation
Since the size of the dataset strongly influences the performance of deep neural networks, 7 it is common practice to augment the data, for example by adding slightly modified copies of existing data. We propose to perform different augmentation techniques depending on the number of available spectra per class: >100: If a class has more than 100 spectra, we do not perform any augmentation as sampling more spectra tends to increase the within-class variation even further.

2-100:
If a class has between 2 and 100 spectra, we add linear combinations between pairs of spectra with randomly simulated coefficients to generate new spectra following Liu et al. 7 .
1: If a class has only one spectrum, linear combinations cannot be used. Instead, we augment the spectrum by adding noise simulated to mimic realistic spectral variation with highest variance near spectral peaks. We use Gaussian noise with amplitude proportional to the standard deviation σ w over a length K sliding window of the first difference d w = s w − s w−1 of the intensities, where s w andŝ w are the original and augmented spectra, and w is the wavenumber index. The first difference is used to indicate the existence of peaks, and the standard deviation σ w can highlight the size and the visibilities of the peaks. The parameters K = 10 and κ = 5 were chosen experimentally to match typical spectral variation.
An example of an original and three augmented spectra is shown in Fig 4. We also experimented with more simple augmentation techniques which performed less well, see S.2 in the supplement.

Ensembling
Rather than using a single model, we use an ensemble of models both to improve the predictive performance and to quantify the predictive uncertainty. 18,29,30 Following Lee et al. 31 , we train these models with the same training data but with different random initialization. We then use the average predictions from the ensemble to make decisions for an unknown spectrum at test time.

Cross-validation
We use a nested cross-validation that divides the data into three sets: a training set used to fit the neural network models, a validation set to optimize model hyperparameters, and a test set to give an unbiased evaluation of the performance.

Test set
For the Mineral and Organic datasets, because the number of spectra per class is limited, we randomly select one spectrum per class to form the test set and use the remaining spectra as the training/validation set. We perform this randomized leave-one-out four times to generate different data splits to demonstrate the robustness of the model. As for the Bacteria dataset, we simply follow 4 and use the existing test dataset to demonstrate the performance.

Validation set
The remaining data, not used for the test set, is used to form the training and validation sets.
For the Mineral and Organic dataset, since many classes have only a few observed spectra, it is not data-efficient to hold out observations for hyperparameter optimization. Therefore, instead of selecting a small set from the training dataset to formulate the validation pairs, we propose to simulate spectra by adding noise onto the measured spectra and randomly shifting them w.r.t. wavenumber. For the positive pairs, we shift the spectra either up or down with a very small wavenumber, 3 cm −1 , as this is a common augmentation technique to simulate more spectra. 5,7 For the negative pairs, we shift the spectrum either up or down with a larger random wavenumber, following a normal distribution with mean 150 cm −1 and standard deviation 50 cm −1 . Since the scale of the shifting is relatively large, the peaks in the original and simulated spectra are at different locations and can thus serve as negative pairs. We simulate a new set of shifting parameters at each validation step to increase the diversity of the validation pairs. For the Bacteria dataset which is relatively large, we simply select 20 spectra (10 from the reference dataset and 10 from the reference-finetune dataset) from each class to formulate the validation positive and negative pairs.

Conformal prediction
Understanding the relationship between matching accuracy and uncertainty level helps us know when the model is likely to make mistakes; however, in practice we need to make decisions based on the model output, and simply knowing when the model is likely to make mistakes is often not enough. Using the method of conformal prediction, 32-35 instead of outputting a single prediction, we can produce prediction set of varying size, that provably cover the true class with a high user-specified probability (i.e., 99%).
Let X n denote a data point and Y n the corresponding true label. Conformal prediction 22,23 aims to construct a predictive set C n,α (X n ) such that: where α ∈ (0, 1) specifies the desired coverage level. To find the predictive set C, the procedure uses a threshold τ on the similarity scores to determine which class predictions should be included. This threshold is estimated using independently, identically distributed hold-out samples, 22 and in our setting we use the validation data. Since for the Mineral and Organic experiments the validation data is not independent (it was created by data augmentation) we limit our analysis to the Bacteria dataset.
To find the threshold τ given the desired coverage level α, we first compute the maximum similarity score for the correct class on the validation data. We then select τ as α th quantile, such that the 1 − α proportion of the validation data have a similarity score greater than τ .
From the assumption that the validation and test data are identically distributed, we then have that confidence sets computed using τ to threshold the similarity score will have the desired coverage.
To prevent producing an empty predictive set in the cases where none of the predictions have probabilities that are higher than the threshold τ , we always include the top-1 prediction in C. To quantitatively evaluate the performance of conformal prediction, we calculate the empirical coverage, which measures the percentage of having the true class in the predictive set C.

Results and discussion
We first quantitatively and qualitatively evaluate the predictive performance of the proposed method on all the datasets. Next, we demonstrate the relation between the predicted similarity score and matching accuracy. Finally, we quantitatively show the benefit of producing adaptive prediction sets using conformal prediction on the Bacteria dataset.

Spectrum identification accuracy
We compute the global average matching accuracy as the proportion of test spectra that are correctly identified, according to the similarity scores averaged across the ensemble. We compare our method with one-nearest-neighbors (1NN) baseline classifier and the existing state-of-the-art deep learning-based approaches. For the 1NN baseline we include Euclidean distance, Manhattan distance, and cosine similarity, as these similarity measures are widely used in commercial software for spectrum matching. 5 Table 2 displays our results together with a 95% confidence interval across the five data splits, computed as ±1.96 σa √ 4 where σ a is the standard deviation of the accuracy. Table 2: Performance together with a 95% confidence interval over four different data splits on all the datasets (%). The number of model parameters are given in parentheses. Our model significantly outperform the existing approaches on most data sets in terms of matching accuracy while using a much smaller model. as they achieve much better performance on the preprocessed datasets. Contrary to that, our method performs best on the raw datasets which indicates that information useful for discriminating between classes may lost in the preprocessing. We observed that 1NN-cosine performs similar to our approach on the Organic (p) dataset. One of the reasons is that some classes (12 out of 72) in the preprocessed dataset contains virtually identical spectra, which could explain why the nearest neighbor and our reference matching approach strongly outperforms the existing work that uses supervised classification. We also achieved a much higher accuracy compared to the reported transferred learning accuracy that is obtained by using another bigger organic dataset as the base learner. 24

Qualitative evaluation
One of the benefits of working with the Siamese network is that we can understand how the decision is made by looking at the matching regions/areas between the target and retrieved spectra as exemplified in Fig. 5.
We notice that there are two scenarios where the model is most likely to match a spectrum wrong. First, it is difficult for the model to identify a spectrum correctly when Intensity (a.u.)  it is very different from the ones in the reference library. For example, the spectra for mineral Parascholzite in the reference library and test set look different due to different measuring wavelengths (785nm and 532nm). In addition, for bacteria S.lugdunensis, the peak at around 1 100 is not present in the input spectrum (see also S.5 in the supplementaty).
These dissimilarities between the spectra from the same class are a limiting factor for the model. Secondly, if two classes have similar chemical compositions, it can also be hard for the model to distinguish between their spectra. For example, the spectra for the minerals AlumNa and AlumK are almost identical. However, most of the time, these classes tend to be matched wrong with the first candidate but correct with the second or third most probable candidates. Therefore, we next investigate how well we can construct predictive sets that are small for easy-to-recognize spectra but larger for hard-to-recognize spectra.

Uncertainty estimation
To understand how well the predicted similarity score reflects the empirical error when evaluated on unseen dataset, we examine the relationship between the model predictions and matching accuracy. We group the predictions in intervals based on the quartiles of the predicted similarity logit score (network output before the sigmoid function) and calculate the average matching accuracy within each interval. The higher the logit score is, the more similar the model believes the input pair of spectra is. Fig. 6 shows the result. We observe that as the similarity score increases, the matching accuracy also increases, which indicates that the similarity score is useful for quantifying the prediction uncertainty. Ba cte ria Figure 6: Percentage of the correct matching at different confidence levels on the test dataset (average over five different data splits). The higher the similarity logit score is, the more confident the model is. As the confidence increases, the percentage of correct matching also increases

Variable-size prediction sets
To demonstrate the benefit of using conformal prediction, we examine the relation between emperical coverage, theoretical coverage, and average prediction set size in Fig. 7. The average set size is the average of the length of the predictive sets over all the test spectra.
A smaller average set size conveys more detailed information and may be more useful in practice since it will take less time to examine each of the predictions from the set.
We observe that as the theoretical coverage increases, the empirical coverage also increases with a moderate increase of the average set size. Compared to using a fixed-size top-M predicting strategy, we can achieve a much higher matching accuracy by increasing the set size using conformal prediction. For example, with an average set size of 2 we increase our accuracy from 83% to 90% and with an average set size of 7, we can correctly identify 99% of the bacteria.  Figure 7: Left: empirical coverage, average set size and the theoretical coverage for Bacteria dataset. As the theoretical coverage increases, the empirical coverage also increase without drastically increasing the set size. Right: conformal prediction set examples with predicted matching probability (p). By using conformal prediction, we can obtain a safer and more trustworthy predictive set for MSSA 3 since the top-2 predictions Group C Strep. and Group A Strep. require different antibiotics (Penicillian) than MSSA 3 (Vancomycin) 1

Conclusion
In this paper, we proposed to use a Siamese network for Raman spectrum matching. Given pairs of spectra as input, the model learns a feature representation that is optimized for matching spectra belonging to the same class. We use an Xception-based architecture in our network that allows the extracted feature to contain the fingerprint characteristics such as locations, shapes, and the number of peaks as much as possible while keeping the model complexity low. We experimentally demonstrate that we match or outperform existing approaches over three publicly available datasets with a similar or much smaller number of parameters. Our method does not require delicate preprocessing steps such as baseline correction, smoothing, or normalization: In fact, it gives the best results on raw data. The model also reliably estimates the uncertainty of the predictions which is useful to signal when the model is likely to make a wrong prediction. We further use a conformal prediction technique to compute prediction sets that cover the true class with a user-defined coverage level (i.e., 99%). This technique is especially critical for sensitive tasks (i.e., the detection of toxic or explosive materials using Raman spectroscopy).
In practical application outside the laboratory, the measured Raman spectra rarly only include a single component. Therefore, our future work will investigate the spectrum matching for mixture components.