## RESEARCH

## Bayesian Gamma-Negative Binomial Modeling of Single-Cell RNA Sequencing Data

Siamak Zamani Dadaneh , Paul de Figueiredo , Sing-Hoi Sze , Mingyuan Zhou 1 3 4 2 and Xiaoning Qian 1*

## Abstract

Background: Single-cell RNA sequencing (scRNA-seq) is a powerful profiling technique at the single-cell resolution. Appropriate analysis of scRNA-seq data can characterize molecular heterogeneity and shed light into the underlying cellular process to better understand development and disease mechanisms. The unique analytic challenge is to appropriately model highly over-dispersed scRNA-seq count data with prevalent dropouts (zero counts), making zero-inflated dimensionality reduction techniques popular for scRNA-seq data analyses. Employing zero-inflated distributions, however, may place extra emphasis on zero counts, leading to potential bias when identifying the latent structure of the data.

Results: In this paper, we propose a fully generative hierarchical gamma-negative binomial (hGNB) model of scRNA-seq data, obviating the need for explicitly modeling zero inflation. At the same time, hGNB can naturally account for covariate effects at both the gene and cell levels to identify complex latent representations of scRNA-seq data, without the need for commonly adopted pre-processing steps such as normalization. Efficient Bayesian model inference is derived by exploiting conditional conjugacy via novel data augmentation techniques.

Conclusion: Experimental results on both simulated data and several real-world scRNA-seq datasets suggest that hGNB is a powerful tool for cell cluster discovery as well as cell lineage inference.

Keywords: single-cell RNA sequencing; Bayesian; hierarchical modeling

## Introduction

Single-cell RNA sequencing (scRNA-seq) has emerged as a powerful tool for unbiased identification of previously uncharacterized molecular heterogeneity at the cellular level [1]. This is in contrast to standard bulk RNA-seq techniques [2], which measures average gene expression levels within a cell population, and thus ignore tissue heterogeneity. Consideration of cell-level variability of gene expressions is essential for extracting signals from complex heterogeneous tissues [3], and also for understanding dynamic biological processes, such as embryo development [4] and cancer [5].

data is challenging. In addition to high gene expression variability due to cell heterogeneity, the excessive amount of zeros in scRNA-seq hinders the application of classical dimensionality reduction techniques such as principal component analysis (PCA). For instance, in real-world datasets, it has been reported that the first or second principal components often depend more on the proportion of detected genes per cell (i.e., genes with at least one read) than on the actual biological signal [6].

A large body of statistical tools developed for scRNA-seq data analysis include a dimensionality reduction step. This leads to more tractable data, from both statistical and computational point of views. Moreover, the noise in the data can be decreased, while retaining the often intrinsically low-dimensional signal of interest. Dimensionality reduction of scRNA-seq

* Correspondence: xqian@ece.tamu.edu

1 Department of Electrical and Computer Engineering, Texas A&amp;M University, College Station, USA

Full list of author information is available at the end of the article

Several existing computational tools adopt explicit zero-inflation modeling to infer the latent representation of scRNA-seq data. Zero-inflated factor analysis (ZIFA) [7] extends the framework of probabilistic PCA [8] to the zero-inflated setting, by modeling the excessive zeros using Bernoulli distributed random variables which indicate the dropout event. Zeroinflated negative binomial-based wanted variation extraction (ZINB-WaVE) [9] directly models the scRNAseq counts using a zero-inflated negative binomial distribution, while accounting for both geneand celllevel covariates. It infers the model parameters using a penalized maximum likelihood procedure.

Despite its popularity, using an explicit zero-inflation term may place unnecessary emphasis on the zero counts, leading to complication in discovering the latent representation of scRNA-seq data. In this paper, we propose a hierarchical gamma-negative binomial (hGNB) model to both perform dimensionality reduction and adjust for the effects of the gene- and cell-level confounding factors simultaneously. Exploiting the hierarchical structure, the proposed hGNB model is capable of capturing the high over-dispersion present in the scRNA-seq data. More precisely, we factorize the logit of the negative-binomial (NB) distribution probability parameter to identify latent representation of the data. In addition to factorization, linear regression terms are also included in that logit function to adjust for the impact of covariates.

In hGNB, a gamma distribution with varying rate parameter is used to model the cell dependent dispersion parameter of the NB distribution. The cell-level dispersion serves as a means of representing the prevalence of the dropout events. For instance, cells that are sequenced deeply will naturally include less droppedout genes with zero counts, and thus this will be reflected in the cell specific dispersion parameter of NB distribution.

We follow a Bayesian framework, similar to bulk RNA-seq setting [10-12], and derive closed-form Gibbs sampling update equations for the model parameters of hGNB, by exploiting sophisticated data augmentation techniques. More specifically, we apply the data augmentation technique of [13] (2015) for the NB distribution, and the Polya-Gamma distributed auxiliary variable technique of [14] (2013) for the closedform inference of regression coefficients and also latent factor parameters, removing the need for non-trivial Metropolis-Hastings correction steps [15]. Experimental results on several real-world scRNA-seq datasets demonstrate the superior performance of hGNB to identify cell clusters, especially in complex settings, and also its potential application in cell lineages inference.

## Methods

## hGNB model

In this section we present the hierarchical gammanegative binomial (hGNB) model for factor analysis of scRNA-seq data. The graphical representation of hGNB is shown in Figure 1. The parameters of the hGNB model with their interpretations in the context of scRNA-seq experiments are presented in Table 1. Let n vj denote the number of sequencing reads mapped to gene v ∈ { 1 , ..., V } in the cell j ∈ { 1 , ..., J } . Under the hGNB model, gene counts are distributed according to a negative binomial (NB) distribution:

$$\vdash \quad n _ { v j } \sim N B ( r _ { j }, p _ { v j } ),$$

where r j and p vj are dispersion and probability parameters of NB distribution, respectively. The probability mass function (PMF) of this distribution can be expressed as f N ( n vj ) = Γ( n vj + r j ) n vj !Γ( r j ) p n vj vj (1 -p vj ) r j , where Γ( ) is the gamma function. ·

Data from scRNA-seq experiments exhibit high variability between different cells, even for genes with medium or high levels of expression. To capture this variability, we impose a gamma prior on the cell-level dispersion parameters as

$$r _ { j } \sim \text{Gamma} ( e _ { 0 }, 1 / h ),$$

where for simplification, the hyper-parameter e 0 is set to 0.01 in our experiments, and the rate h is learned during the Gibbs sampling inference, presented in the following section. This hierarchical prior on the dispersion parameter, enhances the flexibility of NB distribution to capture the high over-dispersion of scRNAseq counts, without the need for explicit zero-inflation modeling.

To account for various technical and biological effects common in scRNA-seq technologies, we impose a regression model on the logit of NB probability parameter as

$$\frac { \alpha } { \vdash } \quad \psi _ { v j } = \log i t ( p _ { v j } ) = \beta _ { v } ^ { T } x _ { j } + \delta _ { j } ^ { T } z _ { v } + \phi _ { v } ^ { T } \theta _ { j }. \quad ( 3 )$$

The three terms in the summation are described below.

In the first term, x j is a known vector of P covariates for cell j and β v is the regression-coefficient vector adjusting the effect of covariates on gene v . The covariate vector x j can represent variations of interest, such as cell types, or unwanted variations, such as batch effects or quality control measures. An intercept term can also be included in these cell-level covariates to account for gene dependent baseline expressions.

In the second term, z v is a vector of Q covariates for gene v , representing gene length or GC-content for example [16], and δ j is its associated regressioncoefficient vector. We also include a fixed intercept element in z v to account for cell-specific expressions, such as the size factors representing differences in sequencing depth.

In the third term, φ θ T v j corresponds to the latent factor representation of the count n vj , after accounting for the effects of geneand cell-level covariates. More precisely, the unknown K × 1 vector φ v contains the factor loading parameters which determine the association between genes and latent factors. Moreover,

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00650v1-figure-c32c2ea0-0200-4e95-9b9d-e9e56a237ff4.jpg)

the unknown K × 1 vector θ j encodes the popularity of the K factors in the expression of cell j .

We place independent zero-mean normal distributions on the components of the regression coefficient parameters β v and δ j as

$$\beta _ { v } & \ \sim \ \prod _ { p = 1 } ^ { P } N ( \beta _ { v p } ; 0, \alpha _ { p } ^ { - 1 } ), \quad & \text{in } \text{thm} \\ \delta _ { j } & \ \sim \ \prod _ { q = 1 } ^ { Q } N ( \delta _ { j q } ; 0, \eta _ { q } ^ { - 1 } ), \quad & \text{$(4)$ $Samp} \\ \quad & \text{augm} \\ N B d i$$

where α p and η q are precision parameters of the normal distributions and gamma priors are imposed on them. These priors are known as automatic relevance determination (ARD), which are effective tools for pruning large numbers of irrelevant covariates [17, 18]. In addition, by assuming identical precision for components of the regression coefficients across all genes or samples, hGNB borrows statistical strengths to infer these precision parameters.

We impose independent normal priors on latent factor loading and score parameters φ v and θ j :

$$\phi _ { v } & \ \sim \ \ N ( \phi _ { v } ; 0, I _ { K } ), \quad & \text{ erate} \\ \theta _ { j } & \ \sim \ \prod _ { k = 1 } ^ { K } N ( \theta _ { j k } ; 0, \gamma _ { k } ^ { - 1 } ). \quad & \text{ (5)} \quad & \text{ (.}$$

Note that the posterior for these terms is not generally independent or normal, but accounts for the statistical dependence as reflected in the data.

We complete the model by imposing a gamma prior on the precision parameters of normal distributions, and also the rate parameter of gamma distributions. Specifically, throughout the experiments, we set both the shape and rate of these gamma priors to 0.01.

## Inference via Gibbs sampling

In this section, we provide an efficient inference algorithm that adopts data augmentation techniques tailored to our hGNB model. Algorithm 1 summarizes all the steps in the Gibbs sampling algorithm.

Sample dispersion parameter We start with the data augmentation technique developed for inferring the NB dispersion parameter [13]. More precisely, the negative binomial random variable n ∼ NB( r, p ) can be generated from a compound Poisson distribution as

$$\vdash \\ \overline { g } \\ \overline { s } \\ \overline { s }$$

where u ∼ Log( ) corresponds to the logarithmic ranp dom variable [19], with the PMF f U ( u ) = -p u u ln(1 -p ) , u ∈ { 1 2 , , ... } . As shown in [13], given n and r , the distribution of glyph[lscript] is a Chinese Restaurant Table (CRT) distribution, ( glyph[lscript] | n, r ) ∼ CRT( n, r ), which can be generated as glyph[lscript] = ∑ n t =1 b , b t t ∼ Bernoulli( r r + t -1 ).

Utilizing this augmentation technique, for each observed count n vj , an auxiliary count is sampled as

$$( \ell _ { v j } | - ) \sim \text{CRT} ( n _ { v j }, r _ { j } ).$$

Using gamma-Poisson conjugacy, the cell-dependent dispersion parameters are updated as

$$\text{rior} _ { \text{ons} } \quad ( r _ { j } | - ) \sim \text{Gamma} \left ( e _ { 0 } + \sum _ { v } \ell _ { v j }, \frac { 1 } { h - \sum _ { v } \ln ( 1 - p _ { v j } ) } \right ).$$

Table 1 Parameters of the hierarchical gamma-negative binomial (hGNB) model and their interpretations in the context of scRNA-seq data. The inputs of hGNB are gene counts n vj and vector of cell- and gene-level covariates x j and z v .

| Parameter   | Constraint                   | Interpretation                                     |
|-------------|------------------------------|----------------------------------------------------|
| r j         | r j > 0                      | expression heterogeneity of genes in sample j      |
| φ vk        | ∑ V v =1 φ vk = 1 , φ vk > 0 | gene-latent factor association                     |
| θ jk        | θ kj > 0                     | popularity of factor k in sample j                 |
| β vp        | β vp ∈ R                     | impact of cell covariate p on expression of gene v |
| δ jq        | β vp ∈ R                     | impact of gene covariate q on expression of cell j |

(7)

Sample regression coefficients For the regression coefficients modeling potential covariate effects, the lack of conditional conjugacy precludes immediate closedform inference. Therefore we adopt another data augmentation technique, specifically designed for hGNB, to infer the regression coefficients β v and δ j , relying on the Polya-Gamma (PG) data augmentation [14, 20].

Denote ω vj as a random variable drawn from the PG distribution as ω vj ∼ PG( n vj + r , j 0) . Since E ω vj [exp( -ω vj ψ 2 vj / 2)] = cosh ( n vj + r j ) ( ψ 2 vj / 2), the likelihood of ψ vj in (3) can be expressed as

$$\mathcal { L } ( \psi _ { v j } ) & \in \frac { ( e ^ { \psi _ { v j } } ) ^ { n _ { v j } } } { ( 1 + e ^ { \psi _ { v j } } ) ^ { n _ { v j } + r _ { j } } } & & ( \theta _ { \cdot } \\ & \in \exp \left ( \frac { n _ { v j } - r _ { j } } { 2 } \psi _ { v j } \right ) \mathbb { E } _ { \omega _ { v j } } [ \exp ( - \omega _ { v j } \psi _ { v j } ^ { 2 } / 2 ) ]. \quad \text{where } 2 \\ ( 8 ) & \quad \mu _ { j } ^ { ( \theta ) } = \cdot$$

Exploiting the exponential tilting of the PG distribution in [14], we draw ω vj as

$$( \omega _ { v j } | - ) \sim \text{PG} ( n _ { v j } + r _ { j }, \psi _ { v j } ).$$

Given the values of the auxiliary variables ω vj for j = 1 , ..., J and the prior in (4), the conditional posterior of β v can be updated as

$$( \beta _ { v } | - ) \sim N ( \mu _ { v } ^ { ( \beta ) }, \Sigma _ { v } ^ { ( \beta ) } ), & & ( 1 0 ) & & \gamma$$

$$& \text{where $\Sigma_{v}^{(\beta)}=\left(\text{diag}(\alpha_{1},\dots,\alpha_{P})+\sum_{j}\omega_{vj}\boldsymbol x_{j}\boldsymbol x_{j}\right)^{-1}$ and} \quad \text{Fina} \\ & \mu _ { v } ^ { ( \beta ) } = \Sigma _ { v } ^ { ( \beta ) } \left [ \sum _ { j } \left ( \frac { n _ { v j - r _ { j } } } { 2 } - \omega _ { v j } ( \delta _ { j } ^ { T } z _ { v } + \phi _ { v } ^ { T } \boldsymbol \theta _ { j } ) \right ) x _ { j } \right ]. \quad \text{be up}$$

A similar procedure can be followed to derive the conditional updates for cell-level regression coefficients as

$$( \delta _ { j } | - ) \sim N ( \mu _ { j } ^ { ( \delta ) }, \Sigma _ { j } ^ { ( \delta ) } ), & & ( 1 1 ) & & \text{Resu}$$

$$& \text{where } \Sigma _ { j } ^ { ( \delta ) } = \left ( \text{diag} ( \eta _ { 1 }, \dots, \eta _ { Q } ) + \sum _ { v } \omega _ { v j } z _ { v } z _ { v } ^ { T } \right ) ^ { - 1 } \text{ and } \quad \text{$two$ ev} \\ & \mu _ { j } ^ { ( \delta ) } = \Sigma _ { j } ^ { ( \delta ) } \left [ \sum _ { v } \left ( \frac { n _ { v j } - r _ { j } } { 2 } - \omega _ { v j } ( \beta _ { v } ^ { T } x _ { j } + \phi _ { v } ^ { T } \boldsymbol \theta _ { j } ) \right ) z _ { v } \right ]. \quad \text{$ponent}$$

Sample latent factor parameters Using the likelihood function in (8) and the priors in (5), we can derive closed-form update steps for factor loading and score parameters. More specifically, the full conditional for factor loading φ v is a normal distribution:

$$\mathfrak { i }, \quad \ \ ( \phi _ { v } | - ) \sim N ( \mu _ { v } ^ { ( \phi ) }, \Sigma _ { v } ^ { ( \phi ) } ),$$

$$\int _ { \substack { 0 \text{} } } \, \begin{array} { c } \, \\ \, \\ \, \\ \end{array} } \, \begin{array} { c } \, \\ \, \\ \end{array} } \, \begin{array} { c } \, \\ \, \\ \end{array} } \, \begin{array} { c } \, \\ \, \\ \end{array} } \, \begin{array} { c } \, \\ \, \\ \end{array} } \, \begin{array} { c } \, \\ \, \\ \end{array} } \, \begin{array} { c } \, \\ \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \\ \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \left ( I _ { K } + \sum _ { j } \, \omega _ { v j } \, \theta _ { j } \, \theta _ { j } ^ { T } \right ) ^ { - 1 } \, \text{ and } \, \mu _ { v } ^ { ( \phi ) } \, \ = \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } & \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array}\, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \end{array} } \, \begin{array} { c } \, \ \\ \end{array} } \, \begin{array} { c } \, \end{array} } \begin{array} { c } \, \end{array} } \begin{array} { c } \, \end{array} } \begin{array} { c } \, \end{array} } \begin{array} { c } \, \end{array} } \begin{array} { c } \, \end{array} } \begin{array} { c } \, \end{array} } \begin{array} { c } \, \end{array} } \begin{array} { c } \, \end{array} } \begin{array} { c } \, \end{array} } \end{array} } \begin{array} { c } \, \end{array} } \begin{array} { c } \end{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \begin{array} } \end{array} } \begin{array} } \begin{array} } \end{array} } \begin{array} } \begin{array} } \begin{array} \end{array} } \begin{array} } \end{array} } \begin{array} } \begin{array} \end{array} } \begin{array} } \begin{array} } \begin{array} \end{array} \end{array} } \begin{array} } \begin{array} } \end{array} } \begin{array} \end{array} } \begin{array} \end{array} } \begin{array} } \end{array} } \begin{array} \end{array} } \begin{array} \end{array} } \begin{array} \end{array} } \begin{array} \end{array} } \begin{array} \end{array} \end{array} } \begin{array} \end{array} } \begin{array} \end{array} \end{array} \begin{array} \end{array} } \begin{array} \end{array} \end{array} \begin{array} \end{array} \end{array} \end{array} \begin{array} \end{array} \end{array} \begin{array} \end{array} \end{array} \end{array} \begin{array} \end{array} \begin{array} \end{array} \begin{array} \end{array} \end{array} \end{array} \begin{array} \end{array} \begin{array} \end{array} \begin{array} \end{array} \end{array$$

The full conditional for factor score θ j is also a normal distribution:

$$( \theta _ { j } | - ) \sim N ( \mu _ { j } ^ { ( \theta ) }, \Sigma _ { j } ^ { ( \theta ) } ),$$

$$\psi _ { v j } ^ { 2 } / 2 ) ]. \begin{array} { c } \text{ where } \Sigma _ { j } ^ { ( \theta ) } = \left ( \text{diag} ( \gamma _ { 1 }, \dots, \gamma _ { K } ) + \sum _ { v } \omega _ { v j } \phi _ { v } \phi _ { v } ^ { T } \right ) ^ { - 1 } \text{ and } \\ ( 8 ) \end{array} \mu _ { j } ^ { ( \theta ) } = \Sigma _ { j } ^ { ( \theta ) } \left [ \sum _ { v } \left ( \frac { n _ { v j } - r _ { j } } { 2 } - \omega _ { v j } ( \beta _ { v } ^ { T } x _ { j } + \delta _ { j } ^ { T } z _ { v } ) \right ) \phi _ { v } \right ].$$

Sample precision and rate The precision parameters of normal distributions in (4) and (5) can be updated using the normal-gamma conjugacy:

$$\Psi _ { \Psi } \times \max \min \min \max \max \max _ { \Psi }. \\ \ell _ { p } & \quad \alpha _ { p } \quad \sim \ \text{Gamma} ( e _ { 0 } + V / 2, \frac { 1 } { f _ { 0 } + \sum _ { v = 1 } ^ { V } \beta _ { v p } / 2 } ), \\ \eta _ { q } & \quad \sim \ \text{Gamma} ( e _ { 0 } + J / 2, \frac { 1 } { f _ { 0 } + \sum _ { v = 1 } ^ { V } \delta _ { j q } / 2 } ), \\ \ell _ { k } & \quad \gamma _ { k } \quad \sim \ \text{Gamma} ( e _ { 0 } + J / 2, \frac { 1 } { f _ { 0 } + \sum _ { v = 1 } ^ { V } \theta _ { j k } / 2 } ) ( 1 4 ) \\ \ell _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ | \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \ \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Phi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi_ | \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } | \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi } \quad \Psi _ { \Psi }$$

Finally, the rate of gamma distribution in (2) can be updated using the gamma-gamma conjugacy with respect to the rate parameter:

$$\mathbf e _ { \mathbf s } \quad h \sim \text{Gamma} ( e _ { 0 } ( 1 + J ), \frac { 1 } { f _ { 0 } + \sum _ { j = 1 } ^ { J } r _ { j } } ). \quad ( 1 5 )$$

## Results

We evaluate our hGNB model on four different sets of real-world scRNA-seq data from different platforms, and compare its performance to those of principal component analysis (PCA), ZIFA [7], and ZINB-WaVE [9].

.

## Algorithm 1 hGNB model inference

Inputs : scRNA-seq counts, design matrix of covariate effects, N Outputs : gene module membership matrix

```
        Inputs: scRNA-seq counts, design matrix of covariate effects, N
                       Outputs: gene module membership matrix
                                                                                                                                                                                                        
        Initialize model parameters
        # Do Gibbs sampling:
        for iter = 1 to N do
            Sample_Lvj using the CRT distribution (eq. (6))
            Update rj using the gamma-Poisson conjugacy (eq. (7))
            Sample auxiliary variables w_vj, using the PG distribution (eq.
```

In the following, We briefly describe these scRNA-seq datasets. To pre-process these datasets when needed, we followed the same procedures as in [9].

V1 dataset . This dataset characterizes more than 1600 cells from the primary visual cortex (V1) in adult male mice, using a set of established Cre lines [21]. A subset of three Cre lines, including Ntsr1-Cre, Rbp4Cre, and Scnn1a-Tg3-Cre, that respectively label layer 4, layer 5, and layer 6 excitatory neurons were selected. We only retained 285 cells that passed the authors' quality control (QC) filters. The dimensionality reduction methods were only applied to the 1000 most variable genes.

S1/CA1 dataset . This dataset characterizes 3005 cells from the primary somatosensory cortex (S1) and the hippocampal CA1 region, using the Fluidigm C1 microfluidics cell capture platform followed by Illumina sequencing [22]. Gene expression is quantified by UMI counts.

mESC dataset . This dataset includes the transcriptome measurement of 704 mouse embryonic stem cells (mESCs), across three culture conditions (serum, 2i, and a2i), using the Fluidigm C1 microfluidics cell capture platform followed by Illumina sequencing [23]. We excluded the samples that did not pass the authors's QC filters, resulting in a total of 169 serum cells, 141 2i cells, and 159 a2i cells. The dimensionality reduction methods were only applied to the 1000 most variable genes.

OE dataset . This data characterizes 849 FACSpurified cells from the mouse OE, using the Fluidigm C1 microfluidics cell capture platform followed by Illumina sequencing [24]. We followed the filtering procedure of [25], and filtered the cells that exhibited poor sample quality, retaining a total of 747 cells.

For all datasets, hGNB was run using 2000 MCMC iterations, where after the first 1000 burn-in iterations, the posterior samples with the highest likelihood were collected as the point estimates of model parameters

Figure 2 Mean-difference (MD) plot for S1/CA1 dataset. The solid red line represents the local regression fit to the data

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00650v1-figure-8532fcef-6c8d-4217-82be-bda4903b0ec0.jpg)

corresponding to latent factors. In the dimensionality reduction analysis below, following [9], for S1/CA1 dataset we set the number of latent factors K = 3, and for V1 and mESC we set K = 2.

## Goodness-of-fit of hGNB model

We have examined the goodness-of-fit of hGNB model on V1, S1/CA1 and mESC datasets, using the meandifference (MD) plots. Figure 2 shows the MD plot for the S1/CA1 dataset, where the y-axis is the difference between observed counts and the expected counts under hGNB, and x-axis is the average of these two sets of counts. The solid red line in this figure, which represents the local regression fit [26] to the data, resides near zero for various average levels. This supports the good fit of hGNB model to the highly over-dispersed scRNA-seq data. Similar trends are observed for V1 and mESC datasets (supplementary materials).

## Capturing zero-inflation

Next we evaluate the performance of hGNB on simulated data based on the zero-inflated NB distribution of [9] to show that hGNB faithfully captures zero inflation without the need of explicit zero-inflation modeling. Specifically, the capability of hGNB to recover true clustering structure of cells under three zero-count prevalence levels with two different total numbers of cells. The parameters of the simulating zero-inflated model were learned based on the S1/CA1 dataset. Genes that did not have at least five reads in at least five cells were filtered out and 1000 genes were then sampled at random for each dataset. The number of latent factors was set to K = 2. To simulate cell clustering, a K -variate Gaussian mixture distribution with three components was fitted to the inferred factor score

parameters, and then for each simulated dataset, factor scores were generated from K -variate Gaussian distributions. By adjusting the value of regression coefficients in the zero-inflation term of ZINB-WaVE model, we generated synthetic datasets with three levels of zero-count percentages as 40%, 60% and 80% (for details refer to [9]). The number of cells were set to J = 100 and J = 1000. For each scenario, including cell numbers and zero-count prevalence (sparsity) levels, we simulated 10 datasets.

We evaluate the performance of our method for the clustering task based on the average silhouette width measure. The silhouette width s j of sample j is defined as

$$s _ { j } = \frac { b _ { j } - a _ { j } } { \max \{ a _ { j }, b _ { j } \} },$$

where a j is the average distance between sample j and all samples in the cluster that it belongs to, and b j is the minimum average distance between sample j and samples in other clusters.

Figure 3 shows the clustering average silhouette width based on the above simulation setup, for different zero-count prevalence levels and cell numbers. In the setting with small sample size, for 40% and 60% zero fractions, hGNB has the best clustering silhouette width, and for the 80% zero fraction its performance is identical to that of ZINB-WaVE. In the setting with moderate sample size, hGNB has the best clustering silhouette width for 40% zero fraction, and for 60% and 80% zero fractions it closely follows the performance of ZINB-WaVE. This suggests that the hierarchical structure of hGNB equips it with the capacity to capture highly over-dispersed count data, even though an explicit zero-inflation term is not included in its model. Also, ZINB-WaVE requires large enough samples to have robust inference results due to the introduction of zero-inflation terms in its model. Finally, ZIFA and PCA have the worst performance, as they normalize the data before learning its latent representation.

## Dimensionality reduction

We applied hGNB to the three scRNA-seq datasets, V1, S1/CA1 and mESC, to assess its power to separate cell clusters in the low dimensional space, and compared it to PCA, ZIFA, and ZINB-WaVE methods. Figure 4 illustrates the projected scRNA-seq expression of profiled cells in the two-dimensional space for S1/CA1 dataset. The proposed hGNB model provides more biologically meaningful latent representations of scRNA-seq gene expressions for S1/CA1 cells, especially compared to PCA and ZIFA that do not model the counts directly. Furthermore, hGNB leads to more separated clusters of cells in the two-dimensional

Figure 3 ( a ) J = 100 , ( b ) J = 1000 . Performance of different methods based on recovering the true cell clusters in synthetic data based on S1/CA1 dataset. Zero-inflated NB model of ZINB-WaVE is used to simulate scRNA-seq data.

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00650v1-figure-4a608586-bd02-4562-9249-94c5ef9adb22.jpg)

space, compared to ZINB-WaVE. Specifically, hGNB distinguishes microglia from endothelial-mural cells, while ZINB-WaVE fails to accomplish this task.

To examine the dimensionality reduction results more carefully, we used the average silhouette width as a measure of goodness for clustering.

Figure 5 shows the average silhouette width of different methods on V1, S1/CA1, and mESC datasets. For PCA and ZIFA, the results on both raw counts and normalized counts are included in this figure. For S1/CA1 dataset, which has the highest number of clusters, the proposed hGNB method outperforms all other methods in terms of clustering average silhouette. For mESC dataset, performance of hGNB is comparable to ZINB-WaVE, and it is significantly better than PCA and ZIFA. For V1 dataset, however, we observe that hGNB, besides PCA applied to raw counts, possess the lowest average silhouette. By further examination of the latent representations of cells for this dataset

Figure 4 Low-dimensional representations of the S1/CA1 dataset. Panels correspond to ( a ) PCA (on total-count normalized data), ( b ) ZIFA (on total-count normalized data), ( c ) ZINB-WaVE, and ( d ) hGNB.

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00650v1-figure-e0e45bfe-857f-40b5-85be-05ceda200757.jpg)

Figure 5 Average silhouette width in scRNA-seq datasets ( a ) S1/CA1, ( b ) mESC, and ( ) V1. Silhouette widths were computed in c the low-dimensional space, using the groupings provided by the authors of the original publications. PCA and ZIFA were applied with both unnormalized (RAW) data and after total count (TC) normalization.

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00650v1-figure-220f3218-3101-4bb6-867c-e26e90ed6288.jpg)

(supplementary materials), we observe that all methods split the Rbp4-Cre KL100 cells into two clusters, one of them located near Scnn1a-Tg3-Cre cells, suggesting the presence of batch effects, which have led to confounding of latent representations [9].

Table 2 Correspondence between identified clusters and cell types in OE dataset.

## Identification of developmental lineages

In addition to characterization of cell types, we further demonstrate the capability of hGNB to derive novel biological insights, by analyzing a set of cells from the mouse olfactory epithelium (OE). The samples were collected to identify the developmental trajectories that generate olfactory neurons (mOSN), sustentacular cells (mSUS), and microvillous cells (MV) [24].

We first performed dimensionality reduction on the OE dataset by applying hGNB with K = 50. Next, we clustered the cells using the low-dimensional factor score parameters θ kj . More specifically, the resampling-based sequential ensemble clustering (RSEC) framework implemented in the RSEC function from the Bioconductor R package clusterExperiment [27] was applied to factor scores, leading to identification of 14 cell clusters. The correspondence between the detected clusters and the underlying biological cell types is presented in Table 2. In addition to these already

| Cell Type                         | Clusters                                        |
|-----------------------------------|-------------------------------------------------|
| GBC mSUS mOSN Immature Neurons MV | cl4,cl9 cl2,cl3,cl5,cl11 cl8,cl12,cl3 cl10 cl14 |

known cell clusters in OE, hGNB is able to detect new clusters, potentially offering novel biological insights.

We further investigated the potential benefit of using the learned latent representation by our proposed hGNB model to infer branching cell lineages and order cells by developmental progression along each lineage. To infer the global lineage structure (i.e., the number of lineages and where they branch), a minimum spanning tree (MST) was constructed on the clusters identified above by RSEC. We used the R package slingshot [28]. Figure 6 illustrates the inferred lineages for the OE dataset, in a two-dimensional space obtained by applying multi-dimensional scaling (MDS) algorithm to the factor scores learned by hGNB. There are three branches in the inferred lineages, with endpoints located in microvillous (MV), mature olfactory

Figure 6 Lineage inference on the OE dataset. The low dimensional data representation derived by hGNB were used to cluster cells by RSEC . The minimum spanning tree (MST) of the derived clusters constructed by slingshot is also displayed.

![image](https://ac.marcodsn.me/arxiv-markdown-images/1908.00650v1-figure-8753fb23-0727-4a19-8ce5-5fa65191ccfe.jpg)

sensory neurons (mOSN), and mature sustentacular (mSUS) cells.

## Conclusions

We propose a hierarchical Bayesian gamma-negative binomial (hGNB) model for extracting low dimensional representations from single-cell RNA sequencing (scRNA-seq) data. hGNB obviates the need for explicit modeling of the zero-inflation prevalent in scRNA-seq count data. Our hGNB can naturally account for covariate effects at both the gene and cell levels, and does not require the commonly adopted pre-processing steps such as normalization. By taking advantage of sophisticated data augmentation techniques, hGNB possesses efficient closed-form Gibbs sampling update equations. Our experimental results on real-world scRNA-seq data demonstrates that hGNB is capable of identifying insightful cell clusters, especially in complex settings.

## Competing interests

The authors declare that they have no competing interests.

## Author's contributions

Conceived the method: SZD, MZ, XQ. Developed the algorithm: SZD, MZ, XQ. Performed the simulations: SZD. Analyzed the results and wrote the paper: SZD, PdF, SHS, MZ, XQ. All authors read and approved the final manuscript.

## Acknowledgements

This work was supported by the National Science Foundation (NSF) Grants 1553281 and 1812641. The authors also would like to thank Texas A&amp;M High Performance Research Computing and Texas Advanced Computing Center for providing computational resources to perform experiments in this paper.

## Author details

1 Department of Electrical and Computer Engineering, Texas A&amp;M University, College Station, USA. 2 McCombs School of Business, The University of Texas at Austin, Austin, USA. 3 Department of Microbial Pathogenesis and Immunology, Texas A&amp;M University , College Station, USA. 4 Department of Computer Science and Engineering, Texas A&amp;M University , College Station, USA.

## References

- 1. Shapiro, E., Biezuner, T., Linnarsson, S.: Single-cell sequencing-based technologies will revolutionize whole-organism science. Nature Reviews Genetics 14 (9), 618 (2013)
- 2. Nagalakshmi, U., Wang, Z., Waern, K., Shou, C., Raha, D., Gerstein, M., Snyder, M.: The transcriptional landscape of the yeast genome defined by rna sequencing. Science 320 (5881), 1344-1349 (2008)
- 3. Macosko, E.Z., Basu, A., Satija, R., Nemesh, J., Shekhar, K., Goldman, M., Tirosh, I., Bialas, A.R., Kamitaki, N., Martersteck, E.M., et al. : Highly parallel genome-wide expression profiling of individual cells using nanoliter droplets. Cell 161 (5), 1202-1214 (2015)
- 4. Deng, Q., Ramsk¨ld, D., Reinius, B., Sandberg, R.: Single-cell rna-seq o reveals dynamic, random monoallelic gene expression in mammalian cells. Science 343 (6167), 193-196 (2014)
- 5. Patel, A.P., Tirosh, I., Trombetta, J.J., Shalek, A.K., Gillespie, S.M., Wakimoto, H., Cahill, D.P., Nahed, B.V., Curry, W.T., Martuza, R.L., et al. : Single-cell rna-seq highlights intratumoral heterogeneity in primary glioblastoma. Science 344 (6190), 1396-1401 (2014)
- 6. Finak, G., McDavid, A., Yajima, M., Deng, J., Gersuk, V., Shalek, A.K., Slichter, C.K., Miller, H.W., McElrath, M.J., Prlic, M., et al. : Mast: a flexible statistical framework for assessing transcriptional changes and characterizing heterogeneity in single-cell rna sequencing data. Genome biology 16 (1), 278 (2015)
- 7. Pierson, E., Yau, C.: Zifa: Dimensionality reduction for zero-inflated single-cell gene expression analysis. Genome biology 16 (1), 241 (2015)
- 8. Tipping, M.E., Bishop, C.M.: Probabilistic principal component analysis. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 61 (3), 611-622 (1999)
- 9. Risso, D., Perraudeau, F., Gribkova, S., Dudoit, S., Vert, J.-P.: A general and flexible method for signal extraction from single-cell rna-seq data. Nature communications 9 (1), 284 (2018)
- 10. Dadaneh, S.Z., Qian, X., Zhou, M.: BNP-Seq: Bayesian nonparametric differential expression analysis of sequencing count data. Journal of the American Statistical Association (in-press, doi:10.1080/01621459.2017.1328358) (2017)
- 11. Dadaneh, S.Z., Zhou, M., Qian, X.: Bayesian negative binomial regression for differential expression with confounding factors. Bioinformatics 34 (19), 3349-3356 (2018)
- 12. Zamani Dadaneh, S., Zhou, M., Qian, X.: Covariate-dependent negative binomial factor analysis of rna sequencing data. Bioinformatics 34 (13), 61-69 (2018)
- 13. Zhou, M., Carin, L.: Negative binomial process count and mixture modeling. IEEE Transactions on Pattern Analysis and Machine Intelligence 37 (2), 307-320 (2015)
- 14. Polson, N.G., Scott, J.G., Windle, J.: Bayesian inference for logistic models using P´lya-Gamma latent variables. J. Amer. Statist. Assoc. o 108 (504), 1339-1349 (2013)
- 15. Chib, S., Greenberg, E.: Understanding the metropolis-hastings algorithm. The american statistician 49 (4), 327-335 (1995)
- 16. Risso, D., Schwartz, K., Sherlock, G., Dudoit, S.: Gc-content normalization for rna-seq data. BMC bioinformatics 12 (1), 480 (2011)
- 17. Wipf, D.P., Nagarajan, S.S.: A new view of automatic relevance determination. In: Advances in Neural Information Processing Systems, pp. 1625-1632 (2008)
- 18. Tipping, M.E.: Sparse bayesian learning and the relevance vector machine. Journal of machine learning research 1 (Jun), 211-244 (2001)
- 19. Johnson, N.L., Kemp, A.W., Kotz, S.: Univariate Discrete Distributions vol. 444. John Wiley &amp; Sons, ??? (2005)
- 20. Zhou, M., Li, L., Dunson, D., Carin, L.: Lognormal and gamma mixed negative binomial regression. In: ICML, pp. 1343-1350 (2012)
- 21. Tasic, B., Menon, V., Nguyen, T.N., Kim, T.K., Jarsky, T., Yao, Z., Levi, B., Gray, L.T., Sorensen, S.A., Dolbeare, T., et al. : Adult mouse cortical cell taxonomy revealed by single cell transcriptomics. Nature neuroscience 19 (2), 335 (2016)

- 22. Zeisel, A., Mu˜oz-Manchado, A.B., Codeluppi, S., L¨nnerberg, P., n o La Manno, G., Jur´us, A., Marques, S., Munguba, H., He, L., e Betsholtz, C., et al. : Cell types in the mouse cortex and hippocampus revealed by single-cell rna-seq. Science 347 (6226), 1138-1142 (2015)
- 23. Kolodziejczyk, A.A., Kim, J.K., Tsang, J.C., Ilicic, T., Henriksson, J., Natarajan, K.N., Tuck, A.C., Gao, X., B¨hler, M., Liu, P., u et al. : Single cell rna-sequencing of pluripotent states unlocks modular transcriptional variation. Cell stem cell 17 (4), 471-485 (2015)
- 24. Fletcher, R.B., Das, D., Gadye, L., Street, K.N., Baudhuin, A., Wagner, A., Cole, M.B., Flores, Q., Choi, Y.G., Yosef, N., et al. : Deconstructing olfactory stem cell trajectories at single-cell resolution. Cell stem cell 20 (6), 817-830 (2017)
- 25. Perraudeau, F., Risso, D., Street, K., Purdom, E., Dudoit, S.: Bioconductor workflow for single-cell rna sequencing: Normalization, dimensionality reduction, clustering, and lineage inference. F1000Research 6 (2017)
- 26. Shyu, W.M., Grosse, E., Cleveland, W.S.: Local regression models. In: Statistical Models in S, pp. 309-376. Routledge, ??? (2017)
- 27. Purdom, E., Risso, D.: clusterexperiment: Compare clusterings for single-cell sequencing. R package version 1 (0) (2017)
- 28. Street, K., Risso, D., Fletcher, R.B., Das, D., Ngai, J., Yosef, N., Purdom, E., Dudoit, S.: Slingshot: Cell lineage and pseudotime inference for single-cell transcriptomics. BMC genomics 19 (1), 477 (2018)
Siamak Zamani Dadaneh Paul de Figueiredo Sing-Hoi Sze Mingyuan Zhou Xiaoning Qian
Bayesian Gamma-Negative Binomial Modeling of Single-Cell RNA Sequencing Data
Background: Single-cell RNA sequencing (scRNA-seq) is a powerful profiling
technique at the single-cell resolution. Appropriate analysis of scRNA-seq data
can characterize molecular heterogeneity and shed light into the underlying
cellular process to better understand development and disease mechanisms. The
unique analytic challenge is to appropriately model highly over-dispersed
scRNA-seq count data with prevalent dropouts (zero counts), making
zero-inflated dimensionality reduction techniques popular for scRNA-seq data
analyses. Employing zero-inflated distributions, however, may place extra
emphasis on zero counts, leading to potential bias when identifying the latent
structure of the data. Results: In this paper, we propose a fully generative
hierarchical gamma-negative binomial (hGNB) model of scRNA-seq data, obviating
the need for explicitly modeling zero inflation. At the same time, hGNB can
naturally account for covariate effects at both the gene and cell levels to
identify complex latent representations of scRNA-seq data, without the need for
commonly adopted pre-processing steps such as normalization. Efficient Bayesian
model inference is derived by exploiting conditional conjugacy via novel data
augmentation techniques. Conclusion: Experimental results on both simulated
data and several real-world scRNA-seq datasets suggest that hGNB is a powerful
tool for cell cluster discovery as well as cell lineage inference.