Learning unsupervised feature representations for single cell microscopy images with paired cell inpainting
Feature representations of cells within microscopy images are critical for quantifying cell biology in an objective way ., Classically , researchers have manually designed features that measure phenomena of interest within images: for example , a researcher studying protein subcellular localization may measure the distance of a fluorescently-tagged protein from the edge of the cell 1 , or the correlation of punctuate proteins with microtubules 2 ., By extracting a range of different features , an image of a cell can be represented as a set of values: these feature representations can then be used for numerous downstream applications , such as classifying the effects of pharmaceuticals on cancer cells 3 , or exploratory analyses of protein localization 1 , 4 ., The success of these applications depends highly on the quality of the features used: good features are challenging to define , as they must be sensitive to differences in biology , but robust to nuisance variation such as microscopy illumination effects or single cell variation 5 ., Convolutional neural networks ( CNNs ) have achieved state-of-the-art performance in tasks such as classifying cell biology in high-content microscopy and imaging flow cytometry screens 6–9 , or segmenting single cells in images 10 , 11 ., A key property driving this performance is that CNNs automatically learn features that are optimized to represent the components of an image necessary for solving a training task 12 ., Donahue et al . 13 and Razavian et al . 14 demonstrate that the feature representations extracted by the internal layers of CNNs trained as classifiers achieve state-of-the-art results on even very different applications than the task the CNN was originally trained for; studies specific to bio-imaging report similar observations about the quality of CNN features 6 , 15 , 16 ., The features learned by CNNs are thought to be more sensitive to relevant image content than human-designed features , offering a promising alternative for feature-based image analysis applications ., However , learning high-quality features with CNNs is a current research challenge because it depends highly on the training task ., For example , autoencoders , or unsupervised CNNs trained to reconstruct input images , usually do not learn features that generalize well to other tasks 17–19 ., On the other hand , classification tasks result in high-quality features 13 , 14 , but they rely on large , manually-labeled datasets , which are expensive and time-consuming to generate ., For example , to address the challenge of labeling microscopy images in the Human Protein Atlas , the project launched a massive crowd-sourcing initiative in collaboration with an online video game , spanning over one year and involving 322 , 006 gamers 20 ., Because advances in high-content throughput microscopy are leading to routine generation of thousands of images 21 , the new cell morphologies and phenotypes discovered and need for integration of datasets 4 may require the continuous update of models ., Unsupervised methods that result in the learning of high-quality features without the use of manual labels would resolve the bottleneck of collecting and updating labels: we would , in principle , be able to learn feature representations for any dataset , without the need for experts to curate , label , and maintain training images ., If obtaining expert-assigned labels for microscopy images is challenging , then obtaining labels for the single cells within these images is even more difficult ., Studying biological phenomena at a single-cell level is of current research interest 22: even in genetically-identical cell cultures , single-cell variability can originate from a number of important regulatory mechanisms 23 , including the cell cycle 24 , stochastic shuttling of transcription factors between compartments 25 , or variability in the DNA damage response 26 ., Thus , ideally , a method would efficiently generate feature representations of single cells for arbitrary datasets using deep learning , without the need for labelled single-cell training data ., In this study , we asked if CNNs could automatically learn high-quality features for representing single cells in microscopy images , without using manual labels for training ., We investigate self-supervised learning , which proposes training CNNs using labels that are automatically available from input images ., Self-supervised learning aims to develop a feature representation in the CNN that is useful for other tasks: the training task is only a pretext , and the CNN may not achieve a useful level of performance at this pretext task ., This differs from weakly-supervised learning , where a learnt network is used directly for an auxiliary task 27 , such as segmenting tumors in histology images using a network trained to predict disease class 28 ., After training , the output of the CNN is discarded , and internal layers of the CNN are used as the features ., The logic is that by learning to solve the pretext task , the CNN will develop features that are useful for other applications ., The central challenge in self-supervised learning is defining a pretext task that encourages the learning of generalizable features 29 ., Successful self-supervised learning strategies in the context of natural images , include CNNs trained to predict the appearance of withheld image patches based upon its context 18 , the presence and location of synthetic artifacts in images 29 , or geometric rotations applied to input images 30 ., The idea is that to succeed at the pretext task , the CNN needs to develop a strong internal representation of the objects and patterns in the images ., When transferred to tasks such as classification , segmentation , or detection , features developed by self-supervised methods have shown state-of-the-art results compared to other unsupervised methods , and in some cases , perform competitively with features learned by supervised CNNs 17 , 29–32 ., Here , we present a novel self-supervised learning method designed for microscopy images of protein expression in single cells ., Our approach leverages the typical structure of these images to define the pretext training task: in many cases , each image contains multiple genetically identical cells , growing under the same experimental condition , and these cells exhibit similar patterns of protein expression ., The cells are imaged in multiple “channels” ( such as multiple fluorescence colours ) that contain very different information ., By exploiting this structure , we define a pretext task that relies only upon image content , with no prior human labels or annotations incorporated ., In our examples , one set of channels represents one or more structures in the cell ( e . g . the cytosol , nucleus , or cytoskeleton ) , and another channel represents proteins of interest that have been fluorescently tagged to visualize their localization ( with a different protein tagged in each image ) ., Then , given both channels for one cell and the structural markers for a different cell from the same image , our CNN is trained to predict the appearance of the protein of interest in the second cell , a pretext task that we term “paired cell inpainting” ( Fig 1A ) ., To solve this pretext task , we reason that the CNN must identify protein localization in the first ( or “source” ) cell and reconstruct a similar protein localization in the second ( or “target” ) cell in a way that adapts to single cell variability ., In Fig 1 , the protein is localized to the nucleoli of human cells–the network must recognize the localization of the protein in the source cell , but also transfer it to the equivalent structures in the target cell , despite differences in the morphology of the nucleus between the two cells ., Thus , by the design of our pretext task , our method learns representations of single cells ., To demonstrate the generalizability of our method , we automatically learn feature representations for datasets of both human and yeast cells , with different morphologies , microscopes and resolutions , and fluorescent tagging schemes ( Fig 1B ) ., We exemplify the quality of the features learned by our models in several use-cases ., First , we show that for building classifiers , the features learned through paired cell inpainting improve in discriminating protein subcellular localization classes at a single-cell level compared to other unsupervised methods ., Next , we establish that our features can be used for unsupervised exploratory analysis , by performing an unsupervised proteome-wide cluster analysis of protein analysis in human cells , capturing clusters of proteins in cellular components at a resolution challenging to annotate by human eye ., Finally , we determine that our features are useful for single-cell analyses , showing that our features can distinguish phenotypes in spatially-variable single cell populations ., We would like to learn a representation for single cells in a collection of microscopy images , I ., We define each image i as a collection of single cells , i = {ci , 1 …ci , n} ., We note that the only constraint on i is that its single cells Ci must be considered similar to each other , so i does not need to be strictly defined as a single digital image so long as this is satisfied; in our experiments , we consider an “image” i to be all fields of view corresponding to an experimental well ., We define single cells to be image patches , so c∈RH×W×Z , where Z are the channels ., We split the images by channel into c = ( x , y ) , where x∈RH×W×Z1 , y∈RH×W×Z2 , and Z1 , Z2 , ⊆ Z . For this work , we assign to Z1 channels corresponding to structural markers , or fluorescent tags designed to visualize structural components of the cell , where all cells in the collection of images have been labeled with the tag ., We assign to Z2 channels corresponding to proteins , or channels where the tagged biomolecule will vary from image to image ., We define a source cell cs , which is associated with a target cell ct satisfying constraints that both cells are from the same image , cs ∈ is , ct ∈ it , is = it , and cs ≠ ct ., Our goal is to train a neural network that will solve the prediction problem yt^=f ( xs , ys , xt ) ∀cs , ct∈I , where yt^ represents the predicted protein channels that vary between images ., For this work , we train the network on the prediction problem by minimizing a standard pixel-wise mean-squared error loss between the predicted target protein yt^ and the actual target protein yt:, L ( yt^ , yt ) =1h∙w∑h , w ( yt^h , w-yth , w ) 2, As with other self-supervised methods , our pretext training task is only meant to develop the internal feature representation of the CNN ., After training , yt^ is discarded , and the CNN is used as a feature extractor ., Importantly , while our pretext task predicts a label yt , we consider our overall method to be unsupervised , because these labels are defined automatically from image content without any human supervision ., One limitation in our pretext task is that some protein localizations are not fully deterministic in respect to the structure of the cell and are therefore challenging to predict given the inputs we define ., In these cases , we observe that the network produces smoothed predictions that we hypothesize are an averaged guess of the localization ., We show an example in Fig 1C; while the source protein is localized to the nucleus in a punctate pattern , the target protein is predicted as a smooth distribution throughout the nucleoplasm ., However , as our inpainting task is a pretext and discarded after training , we are not concerned with outputting fully realistic images ., The averaging effect is likely due to our choice of a mean squared loss function; should more realistic images be desired , different loss functions , such as adversarial losses 33 , may produce better results ., As the goal of our training task is to obtain a CNN that can encode single cell image patches into feature representations , we construct independent encoders for ( xs , ys ) and for xt , which we call the “source cell encoder” and the “target marker encoder” , respectively ., After training with our self-supervised task , we isolate the source cell encoder , and discard all other components of our model ., This architecture allows us to obtain a feature representation of any single cell image patch independently without also having to input target cell markers ., To obtain single cell features , we simply input a single cell image patch , and extract the output of an intermediate convolutional layer in the source cell encoder ., We show a summary of our architecture in Fig 1C ., Following other work in self-supervised learning 17 , 18 , 30 , we use an AlexNet architecture for the source cell encoder , although we set all kernel sizes to 3 due to the smaller sizes of our image patches , and we add batch normalization after each convolutional layer ., We use a smaller number of filters and fewer convolutional layers in the architecture of the target marker encoder; we use three convolutional layers , with 16 , 32 , and 32 filters , respectively ., Finally , for the decoder , we reverse the AlexNet architecture ., The goal of our training is to develop the learned features of our source cell encoder , which will later be used to extract single cell feature representations ., If the network were to utilize bleed-through from the fluorescent marker channels to predict the target marker image , or overfit to the data and ‘memorize’ the target cell images , then the network would not need to learn to extract useful information from the source cell images ., To rule out that our trained models were subject to these effects , we produced inpainting results from a trained model , where we paired source cells with target cells where there is a mismatch between source and target protein localization ., Here , the model has seen both the source and target cells during training , but never the specific pair due to the structure of our training task , as the cells originate from different images ., We qualitatively confirmed that our trained networks were capable of synthesizing realistic results agreeing with the protein localization of the source cell ( S1 Fig ) , suggesting that our models are not trivially overfitting to the target marker images ., Because the training inputs to our self-supervised task consist of pairs of cells , the number of possible combinations is large , as each cell may be paired with one of many other cells ., This property is analogous to data augmentation , increasing the number of unique training inputs to our network ., To sample training inputs , for each epoch , we iterate through every single cell in our training dataset , set it as the source cell , and draw with uniform probability a target cell from the set of all valid possible target cells ., Our pretext task relies on the assumption that protein expression in single cells from the same image is similar ., This assumption is not always true: in the datasets used in this work , some proteins exhibit significant single cell variability in their protein abundance or localization 34 , 35 ., These proteins may contribute noise because paired single cells will have ( unpredictably ) different protein expression patterns and the model will not learn ., Although the Human Protein Atlas documents variable proteins 24 , for our experiments we do not remove these and confirm that our model still learns good features in spite of this noise ., For yeast cells , we used the WT2 dataset from the CYCLoPS database 36 ., This collection expresses a cytosolic red fluorescent protein ( RFP ) in all cells , and tags proteins of interest with green fluorescent protein ( GFP ) ., We use the RFP channel as the structural marker , and the GFP channel as the protein ., To extract single cell crops from the images in this dataset , we segmented our images using YeastSpotter on the structural marker channel 37 , and extract a 64x64 pixel crop around the identified cell centers; we discarded any single cells with an area smaller than 5% or greater than 95% of the image crop , as these are likely artifacts arising from under- or over-segmentation ., We discard any images with fewer than 30 cells ., We preprocessed crops by rescaling each crop to be in the range of 0 , 1 ., These preprocessing operations result in a total of 1 , 165 , 713 single cell image patches grouped into 4 , 069 images ( where each image is 4 fields of view ) , with a total of 4 , 069 of 4 , 138 proteins passing this filter ., We also trained a second different yeast cell model , using a NOP1pr-GFP library previously published by Weill et al . 37 This collection tags all proteins of interest with GFP at the N-terminal of proteins with a NOP1 promoter , and is also imaged in brightfield ., We used the brightfield channel as a structural marker and the GFP channel as the protein ., We extracted and preprocessed single cell crops using the same procedure for the CYCLoPS dataset , and discarded any images with fewer than 30 single cells ., These preprocessing operations result in a total of 563 , 075 single cell image patches grouped into 3 , 067 images ( where each image is 3 fields of view ) , with a total of 3 , 067 of 3 , 916 proteins passing this filter ., Finally , we trained a third yeast cell model , using a dataset previously published by Tkach et al . 38 , a Nup49-RFP GFP-ORF library ., This collection expresses a nuclear pore protein ( Nup49 ) fused to RFP in all cells , and tags proteins of interest with green fluorescent protein ( GFP ) ., We use the RFP channel as the structural marker , and the GFP channel as the protein ., We extracted and preprocessed single cell crops using the same procedure for the CYCLoPS dataset , and discarded any images with fewer than 30 single cells ., These preprocessing operations result in a total of 1 , 733 , 127 single cell image patches grouped into 4 , 085 images ( where each image is 3 fields of view ) , with a total of 4 , 085 of 4 , 149 proteins passing this filter ., For human cells , we use images from version 18 of the Human Protein Atlas 24 ., We were able to download jpeg images for a total of 12 , 068 proteins ., Each protein may have multiple experiments , which image different cell line and antibody combinations ., We consider an image to be of a protein for the same cell line and antibody combination; accordingly , we have 41 , 517 images ( where each image is usually 2 fields of view ) ., We downloaded 3 channels for these images ., Two visualize the nuclei and microtubules , which we use as the structural marker channels ., The third channel is an antibody for the protein of interest , which we use as the protein channel ., To extract single cell crops from this image , we binarize the nuclear channel with an Otsu filter and find nuclei by labeling connected components as objects using the scikit-image package 39 ., We filter any objects with an area of less than 400 pixels , and extract a 512x512 pixel crop around the center of mass of remaining objects ., To reduce training time , we rescale the size of each crop to 64x64 pixels ., We preprocessed crops by rescaling each crop to be in the range of 0 , 1 , and clipped pixels under 0 . 05 intensity for the microtubule and nuclei channels to 0 to improve contrast ., Finally , we remove any images with fewer than 5 cells , leaving a total of 638 , 640 single cell image patches grouped into 41 , 285 images , with a total of 11 , 995 of 12 , 068 proteins passing this filter ., Crop sizes for our datasets ( 64x64 pixels for yeast cells and 512x512 pixels for human cells ) were chosen such that a crop fully encompasses an average cell from each of these datasets ., We note that different image datasets may require different crop sizes , depending on the resolution of the images and the size of the cells ., While each crop is centered around a segmentation , we did not filter crops with overlapping or clumped cells , so some crops may contain multiple cells ., In general , we observed that our models did not have an issue learning to inpaint protein expression from a crop with multiple cells to a crop with a single cell , or vice versa: S1 Fig shows an example of a case where we inpaint protein expression from a crop with two cells to a crop with one cell ., During training , we apply random horizontal and vertical flips to source and target cells independently as data augmentation ., We trained models for 30 epochs using an Adam optimizer with an initial learning rate of 1e-4 ., After training , we extract representations by maximum pooling the output of an intermediate convolutional layer , across spatial dimensions ., This strategy follows previous unsupervised representation extraction from self-supervised learning methods 17 , 30 , which sample activations from feature maps ., To benchmark the performance of features learned using paired cell painting , we obtained features from other commonly-used feature representation strategies ., As classic computer vision baselines for our yeast cell benchmarks , we obtained features extracted using CellProfiler 40 for a classification dataset of 30 , 889 image crops of yeast cells directly from the authors 41 ., These features include measurements of intensity , shape , and texture , and have been optimized for classification performance ., Further details are available from 41 ., We also extracted features from these yeast cell image crops using interpretable expert-designed features by Handfield et al . 1 We followed procedures previously established by the authors: we segmented cells using provided software , and calculated features from the center cell in each crop ., For our transfer learning baselines in our yeast cell benchmarks , we used a VGG16 model pretrained on ImageNet , using the Keras package ., We benchmarked three different input strategies: ( 1 ) we mapped channels arbitrarily to RGB channels ( RFP to red , GFP to green , blue channel left empty ) ; ( 2 ) we inputted each channel separately as a greyscale image and concatenated the representations; ( 3 ) we inputted only the GFP channel as a greyscale image and used this representation alone ., In addition , we benchmarked representations from each convolutional layer of VGG16 , post-processed by maximum pooling across spatial dimensions ( as we did for our self-supervised features ) ., S2 Fig shows classification performance using each layer in VGG16 with the kNN classifier described in our benchmarks , across all three strategies ., In general , we observed that the 3rd input strategy resulted in superior performance , with performance peaking in the 4th convolutional block of VGG16 ., We report results from the top-performing layer using the top-performing input strategy in our benchmarks ., Contrary to previous work in transfer learning on microscopy images by Pawlowski et al . , we extract features from the intermediate convolutional layers of our transferred VGG16 model instead of the final fully-connected layer 15 ., This modification allows us to input our images at their original size , instead of resizing them to the size of the images originally used to train the transferred model ., As our work operates on single cell crops , which are much smaller than the full images benchmarked in previous work ( 64x64 pixels compared to 1280x1024 pixels ) , we found that inputting images at their original size instead of stretching them resulted in performance gains: our top-performing convolutional layer ( block4_conv1 ) with inputs at original size achieved 69 . 33% accuracy , whereas resizing the images to 224x224 and using features from the final fully-connected layer ( as-is , without max-pooling , as described in 15 ) achieves 65 . 64% accuracy ., In addition , we found that extracting features from images at original resolution improves run-time: on our machine , inputting 64x64 crops and extracting features from the best-performing layer was about 16 times faster than inputting resized 224x224 images and extracting features from the final fully-connected layer ., Finally , for the supervised baseline , we used the model and pretrained weights provided by Kraus et al . 6 ., We inputted images as previously described ., To ensure that metrics reported for these features were comparable with the other accuracies we reported , we extracted features from this model and built the same classifier used for the other feature sets ., We found that features from the final fully connected layer before the classification layer performed the best , and report results from this layer ., To compare the performance of various feature representations with our single yeast cell dataset , we built kNN classifiers ., We preprocessed each dataset by centering and scaling features by their mean and standard deviation ., We employed leave-one-out cross-validation and predicted the label of each cell based on its neighbors using Euclidean distance ., S1 Table shows classification accuracy with various parameterizations of k ., We observed that regardless of k , feature representations were ranked the same in their classification performance , with our paired cell inpainting features always outperforming other unsupervised feature sets ., However , k = 11 produced the best results for all feature sets , so we report results for this parameterization ., As classic computer vision baselines for our human cell benchmarks , we curated a set of texture , correlation , and intensity features ., For each crop , we measured the sum , mean , and standard deviation of intensity from pixels in the protein channels , and the Pearson correlation between the protein channel and the microtubule and nucleus channels ., We extracted Haralick texture features from the protein channel at 5 scales ( 1 , 2 , 4 , 8 , and 16 pixels ) ., Finally , as the transfer learning baseline for our human cell benchmarks , we extracted features from the pretrained VGG16 model using the same input strategies and layer as established in our yeast cell benchmark ., To directly measure and compare how a feature set groups together cells with similar localizations in their feature spaces , measured the average pairwise distance between cells in the feature space ., We preprocess single cell features by scaling to zero mean and unit variance , to control for feature-to-feature differences in scaling within feature sets ., Then , to calculate the distance between two single cells c , we use the Euclidean distance between their features f:d ( cx , cy ) =∥fcx-fcy∥2 ., Given two images with the same localization term , we calculate the average distance of all cells in the first image paired with all cells in the second image and normalize these distances to an expectation of the average pairwise distance between images with different localization terms ., A negative normalized average pairwise distance score indicates that the distances are smaller than expectation ( so single cells in images with the same label are on average closer in the feature space ) ., For the Human Protein Atlas images , we restricted our analysis to proteins that only had a single localization term shared by at least 30 proteins , resulting in proteins with 18 distinct localization terms ( as listed in Fig 2B ) ., For each localization term , we calculated average pairwise distances for 1 , 000 random protein pairs with the same localization term , relative to an expectation from 1 , 000 random protein pairs with each of the possible other localization terms ( for a total of 17 , 000 pairs sampled to control for class imbalance ) ., For our experiments controlling for cell line , we also introduce the constraint that the images must be of cells of the same or different cell lines , depending on the experiment ., Because some localization terms and cell line combinations are rare , we did not control for class imbalance and drew 10 , 000 random protein pairs with any different localization terms ( not necessarily each other different term ) , and compared this to 10 , 000 random protein pairs with the same localization term ., Hence , the distances in the two experiments are not directly comparable ., For proteins localized to two compartments , we calculated a score for each cell based upon its distance to the first compartment versus its distance to the second compartment ., To do so , we averaged the feature vectors for all single-cells in images annotated to localize to each compartment alone to define the average features for the two compartments ., Then , for every single-cell , we calculated the distance of the single-cell’s features relative to the two compartments’ averages and take a log ratio of the distance to the first compartment divided by the distance to the second compartment ., A negative number reflects that the single-cell is closer to the first compartment in the feature space , while a positive number reflects that the single-cell is closer to the second compartment ., Code and pre-trained weights for the models used in this work are available at https://github . com/alexxijielu/paired_cell_inpainting ., To assess the quality of feature learned through paired cell inpainting we trained a model for yeast fluorescent microscopy images using paired cell inpainting , on an unlabelled training set comprising of the entire WT2 screen in the CyCLOPS dataset , encompassing 1 , 165 , 713 single cells from 4 , 069 images ., Good features are sensitive to differences in biology , but robust to nuisance variation 5 ., As a first step to understanding if our features had these properties , we compared paired cell inpainting features with other feature sets at the task of discriminating different subcellular localization classes in yeast single cells ., To do this , we made use of a test set of 30 , 889 single cell image patches manually assigned to 17 different protein subcellular localization classes by Chong et al . 41 ., These single cells have been curated from a different image screen than the one we used to train our model , and thus represent an independent test dataset that was never seen by the model during training ., To evaluate feature sets , we constructed a simple kNN classifier ( k = 11 ) and evaluated classification performance by comparing the predicted label of each single cell based on its neighbors to its actual label ., While more elaborate classifiers could be used , the kNN classifier is simple and transparent , and is frequently employed to compare feature sets for morphological profiling 15 , 16 , 42 , 43 ., Like these previous works , the goal of our experiment is to assess the relative performance of various feature sets in a controlled setting , not to present an optimal classifier ., To use the feature representation from a self-supervised CNN , we must first identify which layers represent generalizable information about protein expression patterns ., Different layers in self-supervised models may have different properties: the earlier layers may only extract low-level features , but the later layers may be too specific to production of the pretext task 12 ., For this reason , identifying self-supervised features with a layer-by-layer evaluation of the model’s properties is standard 17–19 , 29 , 30 , 32 ., Since our model has five convolutional layers , it can be interpreted as outputting five different feature sets for each input image ., To determine which feature set would be best for the task of classifying yeast single cells , we constructed a classifier on our test set for the features from each layer independently , as shown in Table 1 ., Overall , we observe that the third ( Conv3 ) and fourth ( Conv4 ) convolutional layers result in the best performance ., Next , we sought to compare our features from paired cell painting with other unsupervised feature sets commonly used for cellular microscopy images ., First , we compared our features with feature sets designed by experts: CellProfiler features 40 that we
Cellular microscopy images contain rich insights about biology ., To extract this information , researchers use features , or measurements of the patterns of interest in the images ., Here , we introduce a convolutional neural network ( CNN ) to automatically design features for fluorescence microscopy ., We use a self-supervised method to learn feature representations of single cells in microscopy images without labelled training data ., We train CNNs on a simple task that leverages the inherent structure of microscopy images and controls for variation in cell morphology and imaging: given one cell from an image , the CNN is asked to predict the fluorescence pattern in a second different cell from the same image ., We show that our method learns high-quality features that describe protein expression patterns in single cells both yeast and human microscopy datasets ., Moreover , we demonstrate that our features are useful for exploratory biological analysis , by capturing high-resolution cellular components in a proteome-wide cluster analysis of human proteins , and by quantifying multi-localized proteins and single-cell variability ., We believe paired cell inpainting is a generalizable method to obtain feature representations of single cells in multichannel microscopy images .
To understand the cell biology captured by microscopy images , researchers use features , or measurements of relevant properties of cells , such as the shape or size of cells , or the intensity of fluorescent markers ., Features are the starting point of most image analysis pipelines , so their quality in representing cells is fundamental to the success of an analysis ., Classically , researchers have relied on features manually defined by imaging experts ., In contrast , deep learning techniques based on convolutional neural networks ( CNNs ) automatically learn features , which can outperform manually-defined features at image analysis tasks ., However , most CNN methods require large manually-annotated training datasets to learn useful features , limiting their practical application ., Here , we developed a new CNN method that learns high-quality features for single cells in microscopy images , without the need for any labeled training data ., We show that our features surpass other comparable features in identifying protein localization from images , and that our method can generalize to diverse datasets ., By exploiting our method , researchers will be able to automatically obtain high-quality features customized to their own image datasets , facilitating many downstream analyses , as we highlight by demonstrating many possible use cases of our features in this study .
learning, fluorescence imaging, engineering and technology, signal processing, social sciences, light microscopy, green fluorescent protein, neuroscience, learning and memory, luminescent proteins, cognitive psychology, microscopy, experimental organism systems, crops, research and analysis methods, imaging techniques, crop science, animal studies, proteins, fluorescence microscopy, agriculture, biochemistry, psychology, image processing, biology and life sciences, yeast and fungal models, cognitive science