summaries: This paper presents a method to reduce the dimensions of resting-state functional connectivity for psychiatry disorders such as autism spectrum disorder, major depressive disorder, and schizophrenia. Their method is based on a conditional variational auto-encoder that utilized the diagnostic label. They evaluated their method on two neuroimaging datasets and obtained clustered 2-dimensional representations of multi psychiatry disorder. The authors present a new conditional VAE, which uses diagnostic information as a supervision signal in two different ways, with the aim to identify small latent embedding spaces for functional brain connectivity that preserve diagnostic information. The authors have potential applications in computational psychiatry in mind, which are indeed extensively discussed in the field (transitioning to a continuous nosological approach to mental illness). They compare their approach on synthetic data to various related approaches and ultimately apply it to two functional connectivity datasets from patients with different psychiatric diagnoses, which appears to give good results.  Their main contribution is technical and lies in defining a new cost function to learn the embedding space. Their work incrementally builds on existing work, but appears to be sufficiently novel. There is also a moderate empirical contribution as they analyze the relationship between MDD, ASD and SCZ. This paper presents a new conditional variational autoencoder (VAE) approach to learn a low dimensional embedding of neuropsychiatric disorders from resting-state functional connectivity data. The proposed approach uses diagnostic information in 2 ways: 1) to cluster samples from the same disorder together via the conditional VAE model, and 2) to separate the clusters using contrastive learning.  The method is compared against other dimension reduction approaches on both a synthetic dataset and 2 real datasets which showed a consistent nosological relation among 3 disorders.  The authors propose a novel variational autoencoder to utilize functional connectivity (FC) features from resting state fMRI (rs-fMRI) scans in order to uncover latent nosological relationships between diverse yet related neuropsychiatric disorders. The autoencoder  seeks to learn a mapping  from high-dimensional FC space to a low-dimensional embedding space that is constrained to preserve pairwise relationships between the diagnostic attributes of the disorders. From a clinical standpoint, this work studies the nosology of complex disorders using a continuous dimensional characterization rather than using categorical and discrete diagnostic labels for supervision.    The authors validate their framework on both synthetic data and two separate clinical rs-fMRI datasets consisting of patients diagnosed with Autism Spectrum Disorder (ASD), Major Depressive Disorder (MDD) and Schizophrenia (SCZ). Their experiments evaluate the consistency of uncovered pairwise nosological relationships inferred from their latent representations.  They demonstrate that their model is capable of reliably inferring a dimensional characterization of multiple brain disorders beyond diagnosis labels. 
abstractText: The unravelled nosological relation among diverse types of neuropsychiatric disorders serves as an important precursor in advocating the dimensional approach to psychiatric classification. Leveraging high-dimensional abnormal resting-state functional connectivity, the crux of mining corresponded nosological relations is to derive a low-dimensional embedding space that preserves the diagnostic attributes of represented disorders. To accomplish this goal, we seek to exploit the available diagnostic information in learning the optimal embedding space by proposing a novel type of conditional variational auto-encoder that incorporates dual utilisation of diagnostic information. Encouraged by the achieved promising results in challenging the conventional approaches in low dimensional density estimation of synthetic functional connectivity features, we further implement our approach on two empirical neuropsychiatric neuroimaging datasets and discover a reliable nosological relation among autism spectrum disorder, major depressive disorder, and schizophrenia.
references.authors: David Adam
references.title: Mental health: On the spectrum
references.venue: Nature News Feature,
references.authors: Verneri Anttila Brendan Bulik-Sullivan Hilary K Finucane Raymond K Walters Jose Bras Laramie Duncan Valentina Escott-Price Guido J Falcone Padhraig Gormley Rainer Malik
references.title: Analysis of shared heritability in common disorders
references.venue: of the brain. Science,
references.authors: Wenjun Bai Changqin Quan Zhi-Wei Luo
references.title: Learning flexible latent representations via encapsulated variational encoders
references.venue: In Proceedings of the AAAI Conference on Artificial Intelligence,
references.authors: Simon Baron-Cohen Sally Wheelwright Richard Skinner Joanne Martin Emma Clubley
references.title: The autism-spectrum quotient (aq): Evidence from asperger syndrome/high-functioning autism, malesand females, scientists and mathematicians
references.venue: Journal of autism and developmental disorders,
references.authors: Aaron T Beck Robert A Steer Roberta Ball William F Ranieri
references.title: Comparison of beck depression inventories-ia and-ii in psychiatric outpatients
references.venue: Journal of personality assessment,
references.authors: Christopher P Burgess Irina Higgins Arka Pal Loic Matthey Nick Watters Guillaume Desjardins Alexander Lerchner
references.title: Understanding disentangling in beta-vae
references.venue: arXiv preprint arXiv:1804.03599,
references.authors: BJ Casey Nick Craddock Bruce N Cuthbert Steven E Hyman Francis S Lee Kerry J Ressler
references.title: Dsm-5 and rdoc: progress in psychiatry research
references.venue: Nature Reviews Neuroscience,
references.authors: Avshalom Caspi Terrie E Moffitt
references.title: All for one and one for all: Mental disorders in one dimension
references.venue: American Journal of Psychiatry,
references.authors: Nick Craddock Michael J Owen
references.title: The kraepelinian dichotomy–going, going but still not gone
references.venue: The British Journal of Psychiatry,
references.authors: Nat Dilokthanakul Pedro AM Mediano Marta Garnelo Matthew CH Lee Hugh Salimbeni Kai Arulkumaran Murray Shanahan
references.title: Deep unsupervised clustering with gaussian mixture variational autoencoders
references.venue: arXiv preprint arXiv:1611.02648,
references.authors: Richard O Duda Peter E Hart
references.title: Pattern classification
references.authors: Allen Frances
references.title: Whither dsm–v? The British
references.venue: Journal of Psychiatry,
references.authors: Jacob Goldberger Geoffrey E Hinton Sam Roweis Russ R Salakhutdinov
references.title: Neighbourhood components analysis
references.venue: Advances in neural information processing systems,
references.authors: Xinyu Guo Kelli C Dominick Ali A Minai Hailong Li Craig A Erickson Long J Lu
references.title: Diagnosing autism spectrum disorder from brain resting-state functional connectivity patterns using a deep neural network with a novel feature selection method
references.venue: Frontiers in neuroscience,
references.authors: Raia Hadsell Sumit Chopra Yann LeCun
references.title: Dimensionality reduction by learning an invariant mapping
references.venue: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06),
references.authors: Anibal Sólon Heinsfeld Alexandre Rosa Franco R Cameron Craddock Augusto Buchweitz Felipe Meneguzzi
references.title: Identification of autism spectrum disorder using deep learning and the abide dataset
references.venue: NeuroImage: Clinical,
references.authors: John E Helzer Helena C Kraemer Robert F Krueger Hans-Ulrich Wittchen Paul J Sirovatka Darrel A Regier
references.title: Dimensional approaches in diagnostic classification: Refining the research agenda for DSM-V
references.venue: American Psychiatric Pub,
references.authors: Martin Heusel Hubert Ramsauer Thomas Unterthiner Bernhard Nessler Sepp Hochreiter
references.title: Gans trained by a two time-scale update rule converge to a local nash equilibrium
references.venue: arXiv preprint arXiv:1706.08500,
references.authors: Geoffrey Hinton Nitish Srivastava Kevin Swersky
references.title: Neural networks for machine learning lecture 6a overview of mini-batch gradient descent
references.venue: Cited on,
references.authors: Quentin JM Huys Tiago V Maia Michael J Frank
references.title: Computational psychiatry as a bridge from neuroscience to clinical applications
references.venue: Nature neuroscience,
references.authors: Diederik P Kingma Jimmy Ba
references.title: Adam: A method for stochastic optimization
references.venue: arXiv preprint arXiv:1412.6980,
references.authors: Diederik P Kingma Max Welling
references.title: Auto-encoding variational bayes
references.venue: arXiv preprint arXiv:1312.6114,
references.authors: Diederik P Kingma Danilo J Rezende Shakir Mohamed Max Welling
references.title: Semi-supervised learning with deep generative models
references.authors: Shinsuke Koike Saori C Tanaka Tomohisa Okada Toshihiko Aso Ayumu Yamashita Okito Yamashita Michiko Asano Norihide Maikusa Kentaro Morita Naohiro Okada
references.title: Brain/minds beyond human brain mri project: a protocol for multi-level harmonization across brain disorders throughout the lifespan
references.venue: NeuroImage: Clinical,
references.authors: Leland McInnes John Healy James Melville
references.title: Umap: Uniform manifold approximation and projection for dimension reduction
references.venue: arXiv preprint arXiv:1802.03426,
references.authors: Grégoire Montavon Alexander Binder Sebastian Lapuschkin Wojciech Samek Klaus-Robert Müller
references.title: Layer-wise relevance propagation: an overview
references.venue: Explainable AI: interpreting, explaining and visualizing deep learning,
references.authors: Charlie Nash SM Ali Eslami Chris Burgess Irina Higgins Daniel Zoran Theophane Weber Peter Battaglia
references.title: The multi-entity variational autoencoder
references.venue: In NIPS Workshops,
references.authors: Aaron van den Oord Oriol Vinyals Koray Kavukcuoglu
references.title: Neural discrete representation learning
references.venue: arXiv preprint arXiv:1711.00937,
references.authors: Matthieu Perrot Denis Rivière Jean-François Mangin
references.title: Cortical sulci recognition and spatial normalization
references.venue: Medical image analysis,
references.authors: Kihyuk Sohn Honglak Lee Xinchen Yan
references.title: Learning structured output representation using deep conditional generative models
references.venue: Advances in neural information processing systems,
references.authors: Saori C Tanaka Ayumu Yamashita Noriaki Yahata Takashi Itahashi Giuseppe Lisi Takashi Yamada Naho Ichikawa Masahiro Takamura Yujiro Yoshihara Akira Kunimatsu
references.title: A multisite, multi-disorder resting-state magnetic resonance image database
references.venue: Scientific data,
references.authors: Tomoki Tokuda Junichiro Yoshimoto Yu Shimizu Go Okada Masahiro Takamura Yasumasa Okamoto Shigeto Yamawaki Kenji Doya
references.title: Identification of depression subtypes and relevant brain regions using a data-driven approach
references.venue: Scientific reports,
references.authors: Jakub Tomczak Max Welling
references.title: Vae with a vampprior
references.venue: In International Conference on Artificial Intelligence and Statistics,
references.authors: Jack Tsai Robert A Rosenheck
references.title: Psychiatric comorbidity among adults with schizophrenia: A latent class analysis
references.venue: Psychiatry research,
references.authors: Arash Vahdat Jan Kautz
references.title: Nvae: A deep hierarchical variational autoencoder
references.venue: arXiv preprint arXiv:2007.03898,
references.authors: Laurens Van der Maaten Geoffrey Hinton
references.title: Visualizing data using t-sne
references.venue: Journal of machine learning research,
references.authors: Stephen J Wood
references.title: Autism and schizophrenia: one, two or many disorders
references.venue: The British Journal of Psychiatry,
references.authors: Neil D Woodward Carissa J Cascio
references.title: Resting-state functional connectivity in psychiatric disorders
references.venue: JAMA psychiatry,
references.authors: Cedric Huchuan Xia Zongming Ma Rastko Ciric Shi Gu Richard F Betzel Antonia N Kaczkurkin Monica E Calkins Philip A Cook Angel Garcia de la Garza Simon N Vandekar
references.title: Linked dimensions of psychopathology and connectivity in functional brain networks
references.venue: Nature communications,
references.authors: Meichen Yu Kristin A Linn Philip A Cook Mary L Phillips Melvin McInnis Maurizio Fava Madhukar H Trivedi Myrna M Weissman Russell T Shinohara Yvette I Sheline
references.title: Statistical harmonization corrects site effects in functional connectivity measurements from multi-site fmri data
references.venue: Human brain mapping,
references.authors: Shengjia Zhao Jiaming Song Stefano Ermon
references.title: Infovae: Information maximizing variational autoencoders
references.venue: arXiv preprint arXiv:1706.02262,
references.authors: Yizhe Zhu Martin Renqiang Min Asim Kadav Hans Peter Graf
references.title: S3vae: Self-supervised sequential vae for representation disentanglement and data generation
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
references.authors: E model
references.title: SRPBS DATABASE & DATA PRE-PROCESSING STEPS The employed UTO and HuShoWa datasets are derived from the SRPBS database, which is consisted of resting-state fMRI scans from 8 different sites. The detailed curation and pre-precessing steps of SRPBS database can be consulted in the work
sections.heading: 1 INTRODUCTION
sections.heading: 1.1 NOSOLOGICAL RELATION AMONG NEUROPSYCHIATRIC DISORDERS
sections.text: Instead of the traditional discrete, categorical view on nosology of multiple neuropsychological disorders (Frances, 2009), an alternative dimensional, continuous view, which suggests all mental illnesses lie along a single low dimensional spectrum (Adam, 2013; Casey et al., 2013; Helzer et al., 2009), uprises to be a more promising perspective to view the relation among diverse disorders. Despite its commonplace in the symptom-based clinical domain (Caspi & Moffitt, 2018), it was not until recently that the finding of shared genetic components across various neuropsychiatric disorders (Anttila et al., 2018) offered the first computational study to reveal the nosological relation among various disorders.
  Aside from the discovered commonalities on the genetic makeup of various disorders, as the restingstate functional connectivity (FC) has proved itself as a valuable biomarker in distinguishing disorder patients from healthy controls in a plethora of computational studies (Woodward & Cascio, 2015), they may also hold great promise for uncovering the complex, continuous relation among different neuropsychiatric disorders. Unfortunately, due to their inherent high dimensionality, it poses a formidable data-analytic challenge in discovering the targeted nosological relation on the highdimensional neuropsychiatric FC feature space. Thence, the crux of unveiling the FC informed nosological relation rests on finding an optimal low-dimensional embedding space to be informative of high-dimensional FC feature space.
sections.heading: 1.2 THE OBJECTIVE OF THIS RESEARCH
sections.text: One straightforward approach is to apply dimensionality reduction techniques to these highdimensional FC features. However, as stated in Huys et al. (2016), without the supplied supervision signals, over-reliance on data-driven methods may not be sufficient to derive the optimal
  low-dimensional embeddings of FC features. Hence, the objective of this research is to seek the utilisation of supervision signals, e.g., diagnostic information, in learning a neural correlate informed low-dimensional embedding space, which allows us to attain the nosological relation among diverse disorders.
  To harness diagnostic information in learning a targeted low-dimensional embedding space, in the following sections, we chiefly propose a novel type of conditional variational auto-encoder (VAE) to incorporate dual utilisation of this information, which involves regularising the learning of embeddings through the introduced implicit clustering effect, and encoding the diagnostic difference among disorders. Based on the demonstrated empirical superiority of our approach in a simulation study, we implemented it in mining a consistent nosological relation among three common disorders across two curated neuropsychiatric FC datasets.
  In summarisation, main contributions of our work include:
  • putting forward a novel type of conditional VAE, which involves two utilisations of diagnostic information in learning an optimal embedding space for high dimensional FC features;
  • mining a consistent nosological relation among autism spectrum disorder, major depressive disorder, and schizophrenia across two curated neuropsychiatric FC datasets.
sections.heading: 2 PREVIOUS WORKS
sections.text: Harnessing the extracted brain features, recent efforts in computational psychiatry are largely acknowledged on developing increasingly complicated classification models to mine discriminative features to aid clinical diagnosis (Guo et al., 2017; Heinsfeld et al., 2018) and discovering the subtypes of a single mental illness via advanced data-driven approaches, e.g., the multi-view clustering method (Tokuda et al., 2018).
  Unfortunately, little attention was paid to unveiling complex nosological relations among diverse types of neuropsychiatric disorders based on brain correlates. Among few attempts, a worth mentioned study that shares a similar motivation with ours is the work of Xia et al. (2018), where dimensions of psychopathology were closely tied to certain common brain features at the macro level, e.g., resting-state networks. However, this finding – in support of the differentiation of disorders on the brain feature level – reveals little on the nosological relationship among disorders, let alone their symptom-based methodology, which does not fully harness the essence of brain features.
  Narrowing our focus on prevailing methodological modifications on VAE, a plethora of variants ranging from the improved generalisation, e.g., beta-VAE (Burgess et al., 2018), Vamp-VAE (Tomczak & Welling, 2018), the diversified generative process, e.g., EVAE (Bai et al., 2019), Multi-entity VAE (Nash et al., 2017), the optimised inference (encoding) process, e.g., Info-VAE (Zhao et al., 2017), the discretised latent space, e.g., vq-VAE (Oord et al., 2017), to the conditional variant and its recent advents, e.g., a semi-supervised VAE (Kingma et al., 2014), NVAE (Vahdat & Kautz, 2020), CVAE (Sohn et al., 2015), and the Gaussian mixture VAE (Dilokthanakul et al., 2016). Falling into the last category, our proposed method can be regarded as a novel type of conditional VAE that embeds a clustering concept. To interested readers, we attach a detailed methodological comparison between our approach and mentioned alternative approaches in A.1. Additionally, rather than the conventional usage in the work of Zhu et al. (2020), our implementation of the contrastive learning is directly applied in the embedding space instead of the original feature space.
sections.heading: 3 DUAL UTILISATION OF DIAGNOSTIC INFORMATION
sections.text: To exploit diagnostic information, i.e., the discrete diagnostic labels c ∈ C, in learning low dimensional latent variables in the embedding space z ∈ Z that are aware of both projected high dimensional FC input features x ∈ X , and their diagnostic attributes, as shown in Figure 1, we delineate dual utilisation of diagnostic information in our proposed conditional VAE.
  1. The provided diagnostic information first serves as observable variables to regularise the learning of z in a novel type of conditional VAE, ensuring the learned (lowdimensional) z to be clustered around close-to-empirical cluster centers.
  2. The z associated diagnostic difference is further encoded in the embedding space via contrastive learning.
  3.1 MANNER-1: REGULARISE THE LEARNING OF z IN A NOVEL CONDITIONAL VAE
  Under the coinage of VAE, an amortised joint inference distribution qφ(z, x) ≡ qφ(z|x)p(x) is commonly employed to approximate the intractable joint generative distribution pθ(z, x) ≡ pθ(x|z)p(z). Aside from the empirical distribution p(x), we further assume p(z) as a Normal distribution, and both qφ(z|x) and pθ(x|z) are conditional Gaussian distributions in which location and scale parameters can be parameterised by complex neural networks. The optimisation objective is acknowledged as the minimisation of the K-L divergence between two joint densities of latent variable and input, i.e., p(z, x) and q(z, x), as: DKL{qφ(z, x)||pθ(z, x)}. To harness provided diagnostic information, we explicitly add discrete variables c ∈ C to represent available diagnostic information; it leads to a conditional VAE (Figure 1 (b)), which the general optimisation objective can be derived as DKL{qφ(z, x, c)||pθ(z, x, c)}. Crucially, to further specify the inner components of two joint densities qφ(z, x, c), and pθ(z, x, c), we adopt a hierarchical encoding process that admits the following derivation: q(φz,φc)(c, z|x) ∝ qφc(c|z)qφz (z|x). In a
  similar vein, a hierarchical decoding process permits: p(θz,θc)(x, z|c) ∝ pθz (x|z)pθc(z|c). As a result, the foregoing joint inference and generative distributions can be derived as: q(φz,φc)(x, z, c) ∝ qφc(c|z)qφz (z|x)p(x) and p(θz,θx)(x, z, c) ∝ pθx(x|z)pθz (z|c)p(c), respectively. From now on, we ignore subscripts for brevity. The general optimisation objective of this novel type of conditional VAE Lcvae with specified hierarchical inference and generative processes can be rewritten into a novel format:
  DKL{qφ(z, x, c)||pθ(z, x, c)} = ∑ c ∫∫ q(c|z)q(z|x)p(x) log q(c|z)q(z|x)p(x) p(x|z)p(z|c)p(c) dzdx
  = Ex∼p(x) [ Ez∼q(z|x)[− log p(x|z)] ] + Ex∼p(x) [ DKL(q(c|z)||p(c)) + ∑ c q(c|z) log q(z|x) p(z|c) ] ,
  (1)
  where we ignore the optimisation irrelevant term log p(x) (an empirical distribution) for brevity. With N number of classes, we define a categorical prior distribution Cat(c|π), π ∈ <N+ for p(c). Three different parameterised Gaussian distributions are further defined as p(x|z) = N (x;µx, I), q(z|x) = N (z;µx, σ2xI), and p(z|c) = N (z;µc, I), respectively, where I is the identity matrix. q(c|z) can be modelled by a multi-layer feedforward soft-max network to produce a categorical representation (logits) for each encoded latent representation.
  The first component in equation 1, i.e., Ez∼q(z|x)[− log p(x|z)], can be realised as the conventional reconstruction loss in a vanilla VAE to ensure the low-dimensional latent variables encoded from high-dimensional FC features. The entire second component, i.e., DKL(q(c|z)||p(c)) +∑ c q(c|z) log q(z|x) p(z|c) , can be understood as the imposed (implicit) clustering effect in the lowdimensional embedding space.
  The first term in the above-mentioned second component, i.e., DKL(q(c|z)||p(c)), serves as a preventative measure to prevent creating a dominant cluster in the face of a class-imbalanced dataset. Intuitively, it encourages the produced categorical representation (logits) c from q(c|z) to be close to its categorical prior distribution p(c). However, since the K-L divergence is an intractable divergence measure, we compute an easy-to-measure alternative quantity, i.e., the cross-entropy of two distributions H(q(c|z), p(c)) in practice. Letting N to indicate the number of input classes, the second term in the second component, i.e., ∑ c q(c|z) log q(z|x) p(z|c) , encourages the learned embedding space to be divided into N different subspaces to ensure each encoded latent representation can be ascribed to 1 of N clusters. Since the loss term that is on the clustering effect can be pinned down to 12 ||z − µc||
  2 (see the derivation in attached A.2), where the cluster centre µc is provided directly from the precomputed empirical cluster mean, the first utilisation of diagnostic information can be regarded as enforcing each encoded embedding z to be closer to its belonged cluster centre µc within each of N clusters.
sections.heading: 3.2 MANNER-2: ENCODING DIAGNOSTIC DIFFERENCE IN THE EMBEDDING SPACE
sections.text: Corresponding to the second manner of utilising diagnostic information, i.e., embedding the diagnostic difference between disorder phenotypes, we resort to contrastive learning (Hadsell et al., 2006) to explicitly model this diagnostic difference. As shown in Figure 1(c), we define a pair of latent vectors−→z1 ,−→z2 ∈ Z to represent two encoded high dimensional brain features, a distance function D to measure the dissimilarity of two latent vectors, and a binary class indicator C = 0, 1 to verify whether latent vectors come from the same disorder phenotype. The contrastive loss function Lclr (Hadsell et al., 2006) in our case can be defined as:
  (1− C)1 2 (D(−→z1,−→z2)) 2 + C 1 2 {max(0,m−D(−→z1,−→z2))} 2, (2)
  where m is a pre-definable margin to weigh the contribution of dissimilar pairs, and we adopt the simple euclidean distance to captureD(−→z1,−→z2), i.e.,D(−→z1,−→z2) = ||
  −→z1−−→z2 ||2. To preserve the authentic diagnostic differences among encoded latent variables, the m is set to 0, i.e., we weigh the equal importance of dissimilar and similar pairs.
  Importantly, bridging the two loss terms, i.e., Lclr and Lcvae together, an overall loss function Ltotal through a convex combination of two terms can be derived as:
  Ltotal = (1− λ)Lcvae + λLclr, (3) where we enforce the equal contribution of two loss terms, i.e., λ = 0.5, to stress the equal importance of two manners in utilising the diagnostic information to derive our targeted embedding space. In attached A.3, we show the effects of differentiated λ in shaping the embedding space for interested readers.
sections.heading: 4 ASSESSING THE IMPORTANCE OF DIAGNOSTIC INFORMATION
sections.text: To assess the importance of diagnostic information in deriving the optimal low-dimensional embedding of neuropsychiatric FC features, we implemented our proposed method in a simulation study, along with the ground truth embeddings. The purpose of this simulation study is twofold. (1) We seek to assess the importance of diagnostic information in recovering ground truth embeddings. (2) We also aim to demonstrate the empirical superiority of our proposed approach over several existing supervised representation learning approaches.
  In preparing for the synthetic high-dimensional FC features, we strictly follow the following threestep generation procedure.
  1. We firstly pre-train a U-map projection function F on a real FC dataset, e.g., the SRPBS multi-disorder brain connectivity database (Tanaka et al., 2021).
  2. 300 low dimensional embeddings are then sampled from three 2D multivariate Gaussians N2(0.8, 1);N2(0.4, 1);N2(0.2, 1). The pair-wise distance relations among three 2D Gaussians are served as the ground-truth relation (see Figure 2).
  3. The inverted projection function F−1 is further utilised to map the foregoing 300 embeddings from the low-dimensional (2D) ground truth embedding space into the highdimensional FC space. In the case of the SRPBS dataset, this produces a [300, 9730] synthetic FC feature matrix.
sections.heading: 4.1 MODELS AND THEIR IMPLEMENTATIONS
sections.heading: 4.1.1 WITHIN OUR APPROACH
sections.text: The crux of implementing our proposed approach is to compute the final loss function equation 3 with its involved loss terms Lcvae in equation 1, and Lclr in equation 2. Aside from the easy computed reconstruction loss, in the remaining components of equation 1, i.e., ∑ c q(c|z) log q(z|x) p(z|c) +
  DKL(q(c|z)||p(c)), the parameters that are pertaining to the optimisation of ∑ c q(c|z) log q(z|x) p(z|c) can be reduced to σ2x, µx, z, and µc. As σx and µx can be parameterised by a feed-forward neural network, it allows the production of z via the re-parameteristion trick Kingma & Welling (2013), i.e., z = ⊗ σx+µx, ∼ N (0, 1). The cluster-wise mean µc can be produced by a simple one layer Gaussian network with supplied supervision signals, e.g., cluster labels in this simulation study. A simple softmax network is sufficient to measure the KL-divergence between q(c|z) and p(c). The second loss term Lclr in equation 2 can be easily captured via adding a margin (distance) based feed-forward neural network on top of encoded latent representations.
  Essentially, to demonstrate the importance of dual utilisation of diagnostic information in our proposed approach, corresponding to above-mentioned two utilisations, three instantiations of our approach are established ranging from the full implementation of our model (Full model) to the mere implementation of Lcvae (Manner-1 model), and Lclr (Manner-2 model). Their layerwise configurations and respective parameter estimation processes are concisely summarised in attached B.1.
sections.heading: 4.1.2 ALTERNATIVE APPROACHES
sections.text: Three categories of alternative methods are also included in this simulation study, ranging from the conventional semi-supervised VAE (S-VAE) (Kingma et al., 2014), the mainstream manifold learning based dimensionality reduction approach, e.g., supervised Umap (S-Umap) (McInnes et al.,
  2018), to the metric learning based neighbourhood component analysis (NCA) (Goldberger et al., 2004). The implementation details of these alternative approaches are also demonstrated in attached B.1.
sections.heading: 4.2 PERFORMANCE EVALUATION
sections.text: As shown in Figure 2, provided with ground truth embeddings and their pairwise distance relations, we target on evaluating the performance of various models regarding the capability of recovering the ground truth embeddings from synthetic high-dimensional FC features. The evaluation metrics include the qualitative visual inspection of yielded embedding spaces, and a quantitative assessment to verdict whether the distance relations among estimated clusters are preserved.
  To quantify the between-cluster distance on the derived low dimensional embedding space, we rely on the pair-wise Wasserstein distance-based Frechet inception distance (FID) based metric (Heusel et al., 2017), i.e., FID(µ1,Σ1,µ1,Σ2), over the conventional L− 1 and L− 2 distance metrics for its consideration on both first and second degree statistics in measuring the discrepancy between two multivariate Gaussians. Its computation can be defined as:
  FID(µ1,Σ1,µ1,Σ2) = |µ1 − µ2| 2 + tr(Σ1 + Σ2 − 2(Σ1Σ2)1/2),
  where µ; Σ are the first and second degree statistics of inferred low-dimensional multivariate Gaussians, i.e., z ∼ N (µ,Σ). On the basis of suggested FID metric to assess the between-cluster relation, the provided ground-truth relation among three clusters can be expressed into the three-way inequality: FID(Cluster1−Cluster3) > FID(Cluster1−Cluster2) < FID(Cluster2−Cluster3).
  Juxtaposed in demonstrated Figure 2, among three instantiations of our approach, the maximum utilisation of diagnostic information, i.e., the Full model, achieved the most promising result on recovering the correct distance relation among three instantiations of our approach, along with the assessed significant pair-wist t-tests among three between-cluster distances in our Full model (see attached C).
  This observation, in conjunction with the consistent result in the second simulation study (see attached D), greatly substantiates the importance of diagnostic information in deriving an optimal low-dimensional space for high-dimensional FC like features. The superiority of our approach over included alternative ones are presented less-distorted low-dimensional embedding space and recovered accurate distance matrix. This observed empirical superiority of our approach assures the further implementation on real neuropsychiatric FC datasets.
sections.heading: 5 MINING THE NOSOLOGICAL RELATION AMONG MULTIPLE DISORDERS
sections.text: After demonstrating the importance of diagnostic information in learning low-dimensional embeddings of FC features, we seek to implement our approach, i.e., the Full model, on curated neuropsychiatric FC datasets to explore nosological relations among disorder phenotypes. In short, high-dimensional FC features of included neuropsychiatric disorders are firstly projected into a common low-dimensional (2D) embedding space, where their pairwise distances are served as the nosological relation in this embedding space.
sections.heading: 5.1 DATASETS & METRICS
sections.text: To acquire a high-quality neuropsychiatric FC dataset that contains multiple types of disorders is a time-consuming task, which demands expertise in diagnosis, and consistency in neuroimaging scanning protocols. The SRPBS multi-disorder brain connectivity database (Tanaka et al., 2021) 1 that pools the fMRI scans of diverse neuropsychiatric patients from 8 different scanning sites with consistent diagnosis guidelines, perfectly serves as our research database.
  UTO dataset From the opted SRPBS database, two datasets were formed to suit our need in studying the nosological relation among multiple neuropsychiatric disorders. The primary dataset comes from the data that were acquired at the University of Tokyo Hospital (AKA., UTO dataset), which is
  1The URL link for this database can be referred to https://bicr-resource.atr.jp/srpbsfc/.
  comprised of fMRI scans from 3 chosen types of clearly diagnosed disorders: autism spectrum disorder (ASD; 10 subjects), major depressive disorder (MDD; 62 subjects), and schizophrenia (SCZ; 35 subjects). These fMRI scans were acquired in the same facility under the identical scanning protocol, which is free from the common threat of site difference in most fMRI studies. The brief introduction of the SRPBS database can be found in attached E.
  The subject-wise brain features, i.e., functional connectivities, were obtained from 10-min restingstate fMRI BOLD signals that underwent the identical preprocessing steps. To compute the subjectwise functional connectivities from these preprocessed BOLD signals, the Brainvisa Sulci Atlas parcellation scheme, which each individual image was divided into 140 regions (Perrot et al., 2011) and the standard Pearson R correlation method were adopted for time series extraction, and the computation of connectivity matrix 2, respectively. Hence, for each subject, 9730 connectivity features were crafted to serve as high dimensional brain features, forming a [107, 9730] brain feature matrix.
  HuShoWa dataset Aside from the main UTO dataset, a replication dataset from the SRPBS database were further curated from the Hiroshima University and ShoWa University (HuShoWa dataset). A total of 307 subjects were included in our HuShoWa dataset, occupying categories of MDD (173 subjects), ASD (115 subjects), and SCZ (19 subjects). Underwent the identical preprocessing steps, and the feature extraction process, we harvested a [307, 9730] FC feature matrix. To minimise the effect of site-difference on acquired fMRI scans in HuShoWa dataset, we resorted to the harmonisation technique, e.g., the combat approach (Yu et al., 2018) to correct site-difference in our input features.
  2To ease the computation, the lower diagonal of connectivity matrix is reserved for connectivity features.
  Validation scheme & performance assessment Since empirical neuropsychiatric FC features are seldom accompanied with ground truth low-dimensional embeddings, it is essential to obtain a strict validation scheme to ensure the reliability of our attained nosological relations. For this, we opt for 10-fold cross validation scheme that demands 10 times running of our model on two empirical datasets.
  In consist with the previous harnessed FID metric in assessing the between-cluster relation, in both UTO and HuShoWa datasets, we applied the FID metric to compute the cluster-wise FID distance on the basis of yielded low-dimensional embeddings of three disorders, e.g., FID(ASD−MDD);FID(ASD−SCZ);FID(MDD−SCZ), their inequality may inform us the nosological relation among ASD, MDD, and SCZ. Over the implemented 10-fold cross validation, multiple paired t-tests in testing the statistical significance of these pair-wise FID distances were also conducted to assess the reliability of attained nosological relations.
sections.heading: 5.2 DISCOVERED NOSOLOGICAL RELATIONSHIPS AMONG ASD, MDD AND SCZ
sections.text: As shown in Figure 3 (left panel), on low-dimensional embedding spaces that are derived from the UTO and HuShoWa datasets, with maximum utilisation of diagnostic information, the learned embeddings are compact, clear-cut clustered in accordance with their represented disorder phenotypes 3.
  In respect of the attained highly consistent embedding spaces (Figure 3 left panel), two similar FID matrices (Figure 3 middle panel), and high statistical power in conducted paired t-tests (Figure 3 right panel) across two curated neuropsychiatric datasets, it unveils us an interestingly between-phenotype relation among the considered ASD, MDD and SCZ, i.e., FID(ASD−MDD) >
  3To interested readers, in attached F, we demonstrate the empirical comparison of the derived embeddings from our approach and several competing alternatives on HuShoWa dataset.
  FID(ASD−SCZ) and FID(ASD−MDD) > FID(MDD−SCZ). This suggests that ASD and MDD may belong to two independent nosology entities, which are loosely connected regarding their large differences in corresponding neural correlates (functional connectivities), whereas the closely linked ASD-SCZ, MDD-SCZ nexuses indicate the potential ’nosological closeness’ between SCZ and the other two disorder phenotypes, which is in line with the previous findings on high rates of comorbidity on SCZ-ASD (Wood, 2017) and SCZ-MDD (Tsai & Rosenheck, 2013).
  Based on this finding, we are able to align these disorder phenotypes on an FC informed dimensional coordinate, allowing ASD and MDD to situate at two ends of this coordinate, in correspondence with two domains of psychopathology, i.e., the neurodevelopmental and affective pathology, respectively (see attached G) (Craddock & Owen, 2010).
sections.heading: 5.3 NEUROSCIENTIFIC INSIGHT
sections.text: Furthermore, since the final projected FCs were represented as two-dimensional features, it is of increasing research interest to see whether these two-dimensional features are in agreement with existing disorder related connectivities in the brain space. For this, harnessing the layer-wise relevance propagation method (Montavon et al., 2019), we have projected each ’cluster centre’ (the most representative feature in each of three clusters) back into the high dimensional FC space, and tentatively identified a significant connection between Caudate and Cuneus that obtains the highest relevance in discriminating three disorders (MDD, SCZ, ASD) among the overall 9730 FCs. However, the validity of this preliminary neuroscientific insight on involved functional connectivity is under the threat of used nonlinear projections (multiple ReLU connections in both inference and generative networks), where the isomorphism between the low-dimensional embedding and high-dimensional FC space is not guaranteed.
  Lastly, to assess the interpretability of learned low-dimensional embedding space in our context, we focus on probing the potential association between learned embeddings and measured clinical or psychological assessments. On the basis of provided ASD related autism spectrum quotient (AQ score) (Baron-Cohen et al., 2001), and MDD related Beck Depression Inventory (BDI score) (Beck et al., 1996) in the HuShoWa dataset, the associations between our derived embeddings and the clinical factors are captured, i.e., AQ score: ρ = 0.533, p < 0.001 and BDI score: ρ = 0.540, p < 0.001, suggesting the potential clinical awareness of derived low-dimensional embeddings.
sections.heading: 6 LIMITATIONS
sections.text: Two limitations of this study are the confined capability of our approach in discerning a novel subtype of a disorder or the nosological overlap between two disorders, and the scarcity of high-quality independent FC datasets that are curated other than the harnessed SRPBS database. The former limitation may be ameliorated through investigating the finer (possibly hierarchical) structure within low-dimensional embeddings of known phenotypes, whereas the latter one may be tackled by the future release of the high-quality imaging datasets from the Brain/Minds beyond MRI project (Koike et al., 2021).
sections.heading: 7 CONCLUSION
sections.text: Unveiling the complex nosological relation among diverse neuropsychiatric disorders with the help of diagnostic information, we advocate a neural correlate based approach, which targets on deriving a neural correlate informed low-dimensional embedding space. To learn low-dimensional embeddings of neural correlates that preserves the diagnostic attributes, dual utilisation of diagnostic information that is incorporated in a novel type of conditional VAE, are proposed in this research. Relying on this method, we discovered a reliable and consistent nosological relation among ASD, MDD, and SCZ across two curated neuropsychiatric FC datasets.
  Undoubtedly, the current approach – at its embryonic stage -- faces several mentioned limitations, a future approach with enhanced interpretability on the obtained low dimensional embedding space, and the inclusion of an independent validation dataset should enable us to attain more reliable nosological relations on various neuropsychiatric disorders to encourage the age-old, discrete categorical view on mental illness to make an exit at a fast pace.
  ETHICS STATEMENT
  Concerning the potential ethnic issues of our propose approach in serving as a clinical application, we raise four concerns on future use of our approach.
  1. Undoubtably, the sole reliance on the discovered nosological relation on basis of our approach is neither optimal nor responsible in the diagnosis procedure. Given the early stage of our approach, harnessing the discovered nosological relation among disorders is still immature to replace established symptom-based diagnosis in clinical psychiatry.
  2. Regarding discovered nosological relations among disorders, given the heterogeneous, complex, and developmental nature of most neuropsychiatric disorders, further validation of such relations is demanded.
  3. Meanwhile, since our probed nosological relation among disorders on 2D embedding space is induced from the computed distance relation, it does not imply any potential signs of comorbidity.
  4. To yield a wide range of social acceptance on using neuroimaging modality in clinical diagnosis, it demands more future validation studies that come from other aspects, e.g., the genetic aspect, and symptom relevance. Solely relying on the neuroimaging modality in clinical diagnosis can be misleading and unethical.
  Unless these raised ethical concerns are addressed, along with additional validation studies and evidence, clinical use of our approach should not be considered and recommended at this moment.
  REPRODUCIBILITY STATEMENT
  To boost the reproducibility of our study, the executable Python code snippet that allows the implementation of our model is contained within the uploaded Zip file. Within the uploaded Zip file, we also include a code snippet that contains the helper function we used in the computation of FID scores between embeddings. The used SRPBS database can be referred to https://bicr-resource.atr.jp/srpbsfc/. The public available version of this referred database can be requested upon the site.
sections.heading: A METHODOLOGICAL DISTINCTIVENESS & DERIVATIONS
sections.text: A.1 METHODOLOGICAL COMPARISON BETWEEN OUR APPROACH AND ALTERNATIVES
  As summarised in following Table 1, the main difference between our approach and NVAE (Vahdat & Kautz, 2020) lies on the derived hierarchical process in defining the decoding and generative process. In NVAE, the hierarchical process can be seen in the sequential production of z, i.e., z = z1, ..., zl; q(z2|x) = q(z2|z1, x). This sequential encoding and decoding procedure allows NVAE to implement feature combinations from z1 to zl. Differ to NVAE, our hierarchical process focuses on the conditional generation of the latent variable z from both discrete variable c, and the input x, i.e., q(c, z|x) ∝ q(z|x)q(c|z). More importantly, with a defined categorical prior distribution for the discrete variable Cat(c|π), π ∈ <N+ for p(c), the N groups of z are encouraged to be mutually independent, i.e., the implicit clustering effect. Such independence is neither the objective nor the outcome of NVAE approach.
  The hierarchical process in our inference model entails the conditional chain q(c|z)q(z|x)p(x), whereas in CVAE (Sohn et al., 2015), such conditional chain is replaced by the conditional density q(z|x, y). In a similar vein, as illustrated in Table 1, the major distinction of our approach to a conventional semi-supervised VAE (Kingma et al., 2014) also lies on their differentiated derivations on inference networks.
  In regards to different instantiations on graphical models, our approach also differs from the Gaussian mixture VAE on both inference and generative networks. In Gaussian mixture VAE (Dilokthanakul et al., 2016), the inference network is defined as q(x,w, z|c), and its corresponded generation network is p(c, x, w, z) ∝ p(x|w, z)p(c|x). Ignoring the Gaussian mixture variable w, the foregoing networks can be seen as q(x, z|c) ∝ q(z|x)q(x|c) and p(x|z)p(c|x). In our networks, these two processes are defined as q(c|z)q(z|x) and p(x|z)p(z|c), respectively. Aside from these differences, we acknowledge the explicit modelling of different Gaussian mixtures w, which cannot be achieved by the current version of our approach.
  In Section 3.1, we put forward a novel type of conditional VAE that allows implicit clustered embeddings. The core component of our loss function is pertaining to log q(z|x)p(z|c) , where we embed the implicit clustering effect. Given two conditional Gaussians to parameterise q(z|x) and p(z|c), respectively, i.e., q(z|x) ∼ N (µx, σ2x), and p(z|c) ∼ N (µc, I), the general optimisation loss term
  for this component can be derived into:
  log q(z|x) p(z|c) = log N (µx, σ2x) N (µc, 1) where N (µx, σ2x) = 1√
  2πσ2x exp
  { −1
  2 ∥∥∥∥z − µxσx ∥∥∥∥2 } , and N (µc, 1) = 1 (2π) exp { −1 2 ‖z − µc‖2 } ,
  = 1
  2 ‖z − µc‖2 −
  1 2 log σ2x − 1 2 ∥∥∥∥z − µ(x)σ(x) ∥∥∥∥2 , as z = µx + ⊗ σx, ∼ N (0, 1)
  = 1
  2 ‖z − µc‖2 −
  1 2 log σ2x + 1 2 || ||2.
  Ignoring the last constant term 1
  2 || ||2, we arrive at
  = 1
  2 ‖z − µc‖2 −
  1 2 log σ2x.
  (4)
  From the following general derivation, it clearly indicates the contribution of µc in producing our targeted low dimensional embeddings z. I.e., from the regularisation viewpoint, µc can be viewed as one type of regulariser to regularise the learning of z.
  A.3 THE IMPORTANCE OF λ IN DETERMINING THE SHAPE OF z
  In the main text, to balance the contribution of two manners, we pre-define the λ = 0.5 to enforce the equal weighting between two manners. However, this λ value be tuned flexibly in determining the shape of each embedded cluster. In the attached Figure 4, through the demonstration on MNIST dataset, we show the effect of differentiated λ in controlling the overlapping degree of encoded clusters. I.e., λ→ 0, clusters of embeddings tend to be overlapped with each other, whereas λ→ 1, encoded clusters are pulling away from each other, in the contribution of the independent, nonoverlapping clustering effect.
sections.heading: B TRAINING CONFIGURATIONS
sections.text: B.1 CONFIGURATIONS IN THE SIMULATION STUDY (SECTION 4.)
  The following two subsections are presented here to demonstrate the default configurations for implementing three instantiations of our proposed approach, i.e., Full model, Manner-1 model, and Manner-2 model, and three alternative models, i.e., S-VAE model, S-Umap model, and NCA model, which were utilised in Section 3 of the main text.
  B.2 CONFIGURATIONS IN THE EMPIRICAL STUDY (SECTION 5.)
  The following table summarises the training configuration of our approach in exploring the nosological relations among various disorder phenotypes in two empirical datasets (Section 4 of the main text). To improve the reliability of the attained nosological relation, we adopt the nearly identical model configurations to implement our model in UTO and HuShoWa datasets, respectively. Regarding the scarce sample sizes of two datasets, the complex cross validation procedure may not be appropriate here. Therefore, in Figure 4 of the main text, the results were yielded from two full datasets.
sections.heading: C STATISTICAL TESTS
sections.text: To assess the statistical significance of attained embedding relation in the above-mentioned simulation study (see Main text Section 4.), we conduct a series of paired t-tests to assess the significance of pair-wise between-cluster FID across six different approaches. As shown in the following Figure 5, our proposed Full model achieved the most superior t-test results among the harnessed 6 approaches, albeit the comparison between C2-C3 and C1-C2 does not achieve the statistical significance. Note here, the observed strong statistical significance between C2-C3 and C1-C2 in NCA, i.e., C2− C3 > C1− C2, contradicts with the ground-truth, i.e., C2− C3 < C1− C2.
sections.heading: D THE SECOND SIMULATION STUDY
sections.text: We attach a supplementary (2nd) simulation study (see the following Figure 6) to further solidify the observed empirical superiority of our approach, i.e., the full model, in recovering low-dimensional embedding of FC-like high-dimensional features. In this attached 2nd simulation study, all set-ups are kept the same with the 1st simulation study (Section 4. main text) except for the non-linear scattering of simulated ground-truth embeddings. Inconsistent with the rendered 1st simulation study, the only model that recovers the ground-truth between-cluster relation is our proposed Full model.
sections.heading: E SRPBS DATABASE & DATA PRE-PROCESSING STEPS
sections.text: The employed UTO and HuShoWa datasets are derived from the SRPBS database, which is consisted of resting-state fMRI scans from 8 different sites. The detailed curation and pre-precessing steps of SRPBS database can be consulted in the work of Tanaka et al. (2021). Here, we highlight some of the key aspects of scanning and data pre-processing. At each site, the subject-wise restingstate fMRI data were obtained from 10-min resting-state fMRI scanning session with the eye-open condition. For data curated in HuShoWa dataset, the imaging data were acquired in a Siemens Spec-
  tra and a Siemens Verio scanner, whereas for the data in UTO dataset, data were acquired from a GE MR750W scanner.
  All resting-state fMRI data underwent the identical pre-processing step include slice-time correlation, realignment, co-registeration, segmentation of T1-weighted structural images, normalisation to Montreal Neurological Institute (MNI) space, and spatial smoothing with an isotropic Gaussian kernel of 6 mm full-width at half-maximum.
sections.heading: F EMPIRICAL COMPARISON OF DERIVED EMBEDDINGS FROM OUR APPROACH AND ALTERNATIVE MODELS
sections.text: Implemented on the same HuShoWa dataset, we empirically evaluate the derived low dimensional embeddings from our approach, and several alternative ones, e.g., neighbourhood component analysis (NCA) (Goldberger et al., 2004), T-SNE (Van der Maaten & Hinton, 2008), Supervised Umap (S-Umap) (McInnes et al., 2018), and a conventional semi-supervised VAE (S-VAE) (Kingma et al., 2014).
sections.heading: G DIMENSIONAL VIEW ON ASD, MDD, & SCZ
sections.text: Demonstrated in the following Figure 8, in accordance with the envisioned dimensional view on diverse neuropsychiatric disorders (Craddock & Owen, 2010), we recast the attained the nosological relation among ASD, MDD, SCZ on the following continua, where ASD and MDD reside at two poles of this continua.
title: DISCOVERING THE NEURAL CORRELATE INFORMED NOSOLOGICAL RELATION AMONG MULTIPLE NEU- ROPSYCHIATRIC DISORDERS THROUGH DUAL UTILISA- TION OF DIAGNOSTIC INFORMATION