text:
\section{Introduction}
\label{section:intro}

Mixture models are used in a wide range of applications, including
population structure \citep{Pritchard_2000},
document modeling \citep{Blei_2003b},
speaker recognition \citep{Reynolds_2000},
computer vision \citep{stauffer1999adaptive},
phylogenetics \citep{pagel2004phylogenetic}, and
gene expression profiling \citep{Yeung_2001}, to name a few prominent examples.
A common issue with finite mixtures is that it can be difficult to choose an appropriate number of mixture components, and many methods have been proposed for making this choice \citep[e.g., ][]{Henna_1985,Keribin_2000,Leroux_1992,Ishwaran_2001,James_2001}.

From a Bayesian perspective, perhaps the most natural approach is to treat the number of components like any other unknown parameter and put a prior on it.  For short, we refer to such a model as a mixture of finite mixtures (MFM).
Several inference methods have been proposed for this type of model \citep{Nobile_1994,Phillips_1996,Richardson_1997,Stephens_2000,Nobile_2007}, the most commonly-used method being reversible jump Markov chain Monte Carlo \citep{green1995reversible,Richardson_1997}.  Reversible jump is a very general technique, and has been successfully applied in many contexts, but it is perceived to be difficult to use, and applying it to new situations requires one to design good reversible jump moves, which can be nontrivial, particularly in high-dimensional parameter spaces.

Meanwhile, infinite mixture models such as Dirichlet process mixtures (DPMs) have become popular, partly due to the existence of generic Markov chain Monte Carlo (MCMC) algorithms that can easily be adapted to new applications \citep{Neal_1992,Neal_2000,MacEachern_1994,MacEachern_1998b,MacEachern_1998,Bush_1996,West_1992,West_1994,Escobar_1995,Liu_1994,Dahl_2003,Dahl_2005,Jain_2004,Jain_2007}.
These algorithms are made possible by the fact that the Dirichlet process has a variety of elegant mathematical properties---an exchangeable partition distribution, the Blackwell--MacQueen urn process (a.k.a.\ the Chinese restaurant process), a random discrete measure formulation, and the Sethuraman--Tiwari stick-breaking representation \citep{Ferguson_1973,Antoniak_1974,Blackwell_1973,aldous1985exchangeability,Pitman_1995,Pitman_1996,Sethuraman_1994,Sethuraman_1981}.

The purpose of this paper is to show that in fact, MFMs typically exhibit many of these same appealing properties---an exchangeable partition distribution, urn/restaurant process, random discrete measure formulation, and in certain cases, a simple stick-breaking representation---and consequently, that many of the inference techniques developed for DPMs can be directly applied to MFMs.
In particular, these properties enable one to do inference in MFMs without using reversible jump.
Interestingly, the key properties of MFMs hold for any choice of prior distribution on the number of components.

There has been a large amount of research on efficient inference methods for DPMs, and an immediate consequence of the present work is that most of these methods can also be used for MFMs.  Since many DPM sampling algorithms (for both conjugate and non-conjugate priors) are designed to have good mixing properties across a wide range of applications---for instance, the Jain--Neal split-merge samplers \citep{Jain_2004,Jain_2007}, coupled with incremental Gibbs moves \citep{MacEachern_1994,Neal_1992,Neal_2000}---this greatly simplifies the use of MFMs in new applications.

This work resolves an open problem discussed by \citet{Green_2001}, who noted that it would be interesting to be able to apply DPM samplers to MFMs:
\begin{quote}
\emph{``In view of the intimate correspondence between DP and [MFM] models discussed above, it is
interesting to examine the possibilities of using either class of MCMC methods for the other
model class. We have been unsuccessful in our search for incremental Gibbs samplers for the
[MFM] models, but it turns out to be reasonably straightforward to implement reversible jump split/merge methods for DP models.''}
\end{quote}

The paper is organized as follows. In the remainder of this section, we motivate this work with an overview of the similarities and differences between MFMs and DPMs, illustrated by a simulation example. In Sections \ref{section:model} and \ref{section:partitions}, we formally define the MFM and show that it gives rise to a simple exchangeable partition distribution closely paralleling that of the Dirichlet process. In Section \ref{section:representations}, we present the P\'olya urn scheme (restaurant process), random discrete measure formulation, and stick-breaking representation for the MFM.  In Section \ref{section:asymptotics}, we establish some asymptotic results for MFMs.  In Section \ref{section:inference}, we show how the properties in Sections \ref{section:partitions} and \ref{section:representations} lead to efficient inference algorithms for the MFM, and in Section \ref{section:empirical}, we apply the model to the galaxy dataset (a standard benchmark) and to high-dimensional gene expression data used to discriminate cancer subtypes.  We close with a brief discussion.

\subsection{Background: Similarities and differences between MFMs and DPMs}
\label{section:comparison}

In many respects, MFMs and DPMs are quite similar, but there are some important differences.
In general, we would not say that one model is uniformly better than the other; rather, one should choose the model which is best suited to the application at hand.

\subsubsection*{Density estimation}
\begin{figure}
\centering
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{estimated_density_n=50_rep=1_MFM.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{estimated_density_n=50_rep=1_DPM.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{estimated_density_n=250_rep=1_MFM.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{estimated_density_n=250_rep=1_DPM.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{estimated_density_n=1000_rep=1_MFM.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{estimated_density_n=1000_rep=1_DPM.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{true_density.png}
\caption{Density estimates for MFM (left) and DPM (right) on increasing amounts of data from a three-component Gaussian mixture (bottom). As $n$ increases, the estimates appear to be converging to the true density, as expected.
See Section \ref{section:simulation} for details.}
\label{figure:density-estimates}
\end{figure}

\begin{figure}
\centering
\includegraphics[trim=0cm 0.5cm 0cm 0.2cm, clip=true, width=0.75\textwidth]{Hellinger_1000.png}
\caption{Hellinger distance to the true density, for MFM (red, left) and DPM (blue, right) density estimates, on data from a three-component Gaussian mixture. For each $n\in\{50,100,250,1000\}$, five independent datasets of size $n$ were used, and the lines connect the averages of the distances for each $n$.
See Section \ref{section:simulation} for details.}
\label{figure:Hellinger}
\end{figure}

For certain nonparametric density estimation problems, both models have been shown to exhibit posterior consistency at the minimax optimal rate, up to logarithmic factors \citep{Kruijer_2010,Ghosal_2007}.
Even for small sample sizes, we observe empirically that density estimates under the two models are remarkably similar.  As a toy example, Figure \ref{figure:density-estimates} compares their density estimates on data from a bivariate Gaussian mixture with three components.  As the amount of data increases, these density estimates appear to be converging to the true density, as expected; indeed, Figure \ref{figure:Hellinger} indicates that the Hellinger distance to the true density is going to zero.

\begin{figure}
\centering
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{sample_clusters_n=250_rep=1_MFM_680.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{sample_clusters_n=250_rep=1_DPM_680.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{sample_clusters_n=250_rep=1_MFM_1000.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{sample_clusters_n=250_rep=1_DPM_1000.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{sample_clusters_n=250_rep=1_MFM_900.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{sample_clusters_n=250_rep=1_DPM_900.png}
\includegraphics[trim=1cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{true_clusters_n=250_rep=1.png}
\caption{Typical sample clusterings from the posterior for the MFM (left) and DPM (right), on $n = 250$ data points from a three-component Gaussian mixture; the bottom plot shows the true component assignments. Note the small extra clusters in the DPM samples (red squares).  Best viewed in color.
See Section \ref{section:simulation} for details.}
\label{figure:clustering}
\end{figure}

\subsubsection*{Clustering}

It seems that more often, mixture models are used for clustering and latent class discovery rather than density estimation. MFMs have a partition distribution that takes a very similar form to that of the Dirichlet process; see Section \ref{section:partitions}.  However, despite this similarity, the MFM partition distribution differs in two fundamental respects.  While the first is widely known, the second is far less often appreciated, and yet is perhaps even more important.

\begin{enumerate}
\item[(1)] The prior on the number of clusters $t$ is very different.  In an MFM, one has complete control over the prior on the number of components $k$, which in turn provides control over the prior on $t$. As the sample size $n$ grows, in an MFM the prior on $t$ converges to the prior on $k$ (in fact, $t$ converges to $k$ almost surely). In contrast, in a Dirichlet process, the prior on $t$ takes a particular parametric form and diverges at a $\log n$ rate.
\item[(2)] Given $t$, the prior on the cluster sizes is very different. In an MFM, most of the prior mass is on partitions in which the sizes of the clusters are all the same order of magnitude, while in a Dirichlet process, most of the prior mass is on partitions in which the sizes vary widely, with a few large clusters and many very small clusters.
\end{enumerate}

See Section \ref{section:asymptotics} for more precise descriptions of (1) and (2) in mathematical terms. (Note that while some authors use the terms ``cluster'' and ``component'' interchangeably, we use \emph{cluster} to refer to a group of data points, and \emph{component} to refer to one of the probability distributions in a mixture model.)

These prior differences carry over to noticeably different posterior clustering behavior.
For instance, Figure \ref{figure:clustering} displays typical clusterings sampled from the posterior, illustrating that DPM samples tend to have tiny ``extra'' clusters, while MFM samples do not.

It should also be mentioned that due to (2), MFMs ``dislike'' partitions with very small clusters, causing incremental Gibbs samplers to mix more slowly (empirically) when $n$ is large, however, this is easily remedied by using split-merge samplers \citep{Jain_2004,Jain_2007}; see Section \ref{section:inference}.

\subsubsection*{Mixing distribution and the number of components}

Assuming that the data is from a finite mixture, it is also sometimes of interest to infer the mixing distribution or the number of components, subject to the caveat that these inferences are meaningful only to the extent that the component distributions are correctly specified and the model is mixture identifiable.  While \citet{Nguyen_2013} has shown that under certain conditions, DPMs are consistent for the mixing distribution (in the Wasserstein metric), \citet{Miller_2014} have shown that the posterior on the number of clusters in a DPM is typically not consistent for the number of components. On the other hand, MFMs are consistent for the mixing distribution and the number of components (for Lebesgue almost-all parameter values) under very general conditions; this is a straightforward consequence of Doob's theorem \citep{Nobile_1994}.  The relative ease with which this consistency can be established for MFMs is due to the fact that in an MFM, the parameter space is a countable union of finite-dimensional spaces, rather than an infinite-dimensional space.

\begin{figure}
\centering
\includegraphics[trim=0cm 0cm 0cm 0cm, clip=true, width=0.49\textwidth]{t_posterior_MFM_1000.png}
\includegraphics[trim=0cm 0cm 0cm 0cm, clip=true, width=0.49\textwidth]{t_posterior_DPM_1000.png} \\
\includegraphics[trim=0cm 0cm 0cm 0cm, clip=true, width=0.49\textwidth]{k_posterior_MFM_1000.png}
\caption{Posterior on the number of clusters $t$ for the MFM (top left) and DPM (top right), and the posterior on the number of components $k$ for the MFM (bottom), on increasing amounts of data from a three-component Gaussian mixture.
See Section \ref{section:simulation} for details.}
\label{figure:tk-posteriors}
\end{figure}

These consistency/inconsistency properties are readily observed empirically---they are not simply large-sample phenomena. As seen in Figure \ref{figure:tk-posteriors}, the tendency of DPM samples to have tiny extra clusters causes the number of clusters $t$ to be somewhat inflated, apparently making the DPM posterior on $t$ fail to concentrate, while the MFM posterior on $t$ concentrates at the true value (see Section \ref{section:clusters-components-relationship}).  In addition to the number of clusters $t$, the MFM also permits inference for the number of components $k$ in a natural way (Figure \ref{figure:tk-posteriors}), while in the DPM the number of components is always infinite.  See Section \ref{section:discussion} for discussion regarding issues with estimating the number of components.

\section{Model}
\label{section:model}

We consider the following well-known model:
\begin{align}
& K\sim p_K, \mbox{ where $p_K$ is a p.m.f.\ on $\{1,2,\dotsc\}$}\notag\\
& (\pi_1,\dotsc,\pi_k)\sim \mathrm{Dirichlet}_k(\gamma,\dotsc,\gamma), \mbox{ given $K = k$}\notag\\
& Z_1,\dotsc,Z_n\stackrel{\mathrm{iid}}{\sim} \pi, \mbox{ given $\pi$}\label{equation:model}\\
& \theta_1,\dotsc,\theta_k\stackrel{\mathrm{iid}}{\sim} H, \mbox{ given $K = k$}\notag\\
& X_j\sim f_{\theta_{Z_j}} \mbox{ independently for $j = 1,\dotsc,n$, given $\theta_{1:K},Z_{1:n}$.}\notag
\end{align}
Here, $H$ is a prior or ``base measure'' on $\Theta\subset\mathbb{R}^\ell$, and $\{f_\theta:\theta\in\Theta\}$ is a family of probability densities with respect to a sigma-finite measure $\zeta$ on $\mathcal{X}\subset\mathbb{R}^d$. (As usual, we give $\Theta$ and $\mathcal{X}$ the Borel sigma-algebra, and assume $(x,\theta)\mapsto f_\theta(x)$ is measurable.) We denote $x_{1:n} = (x_1,\ldots,x_n)$.  Typically, the values $X_1,\dotsc,X_n$ would be observed, and all other variables would be hidden/latent. We refer to this as a \emph{mixture of finite mixtures} (MFM) model.

It is important to note that we assume a symmetric Dirichlet with a single parameter $\gamma$ not depending on $k$. This assumption is key to deriving a simple form for the partition distribution and the other resulting properties.
Assuming symmetry in the distribution of $\pi$ is quite natural, since the distribution of $X_1,\dotsc,X_n$ under any asymmetric distribution on $\pi$ would be the same as if this were replaced by its symmetrized version, i.e., if the entries of $\pi$ were uniformly permuted (although this would no longer necessarily be a Dirichlet distribution).
Assuming the same $\gamma$ for all $k$ is a genuine restriction, albeit a fairly natural one, often made in such models even when not strictly necessary \citep{Nobile_1994,Phillips_1996,Richardson_1997,Green_2001,Stephens_2000,Nobile_2007}.
Note that prior information about the relative sizes of the mixing weights $\pi_1,\dotsc,\pi_k$ can be introduced through $\gamma$---roughly speaking, small $\gamma$ favors lower entropy $\pi$'s, while large $\gamma$ favors higher entropy $\pi$'s.

Meanwhile, we put very few restrictions on $p_K$, the distribution of the number of components. For practical purposes, we need the infinite series $\sum_{k = 1}^\infty p_K(k)$ to converge to $1$ reasonably quickly, but any choice of $p_K$ arising in practice should not be a problem. For certain theoretical purposes---in particular, consistency for the number of components---it is desirable to have $p_K(k)>0$ for all $k\in\{1,2,\dotsc\}$.

For comparison, the Dirichlet process mixture (DPM) model with concentration parameter $\alpha>0$ and base measure $H$ is defined as follows, using Sethuraman's \citeyearpar{Sethuraman_1994} representation:
\begin{align*}
& B_1,B_2,\dotsc\stackrel{\mathrm{iid}}{\sim}\mathrm{Beta}(1,\alpha)\\
& Z_1,\dotsc,Z_n\stackrel{\mathrm{iid}}{\sim} \pi, \mbox{ given } \pi=(\pi_1,\pi_2,\dotsc) \mbox{ where } \pi_i = B_i\textstyle\prod_{j=1}^{i-1} (1 - B_j)\\
& \theta_1,\theta_2,\dotsc\stackrel{\mathrm{iid}}{\sim} H\\
& X_j\sim f_{\theta_{Z_j}} \mbox{ independently for $j = 1,\dotsc,n$, given $\theta_{1:\infty},Z_{1:n}$.}
\end{align*}

\section{Exchangeable partition distribution}
\label{section:partitions}

The primary observation on which our development relies is that the distribution on partitions induced by an MFM takes a form which is simple enough that it can be easily computed.
Let $\mathcal{C}$ denote the unordered partition of $[n]:=\{1,\dotsc,n\}$ induced by $Z_1,\dotsc,Z_n$; in other words, $\mathcal{C} =\{E_i:|E_i|>0\}$ where $E_i =\{j: Z_j = i\}$ for $i\in\{1,2,\dotsc\}$.

\begin{theorem}
\label{theorem:EPPF}
Under the MFM (Equation \ref{equation:model}), the probability mass function of $\mathcal{C}$ is
\begin{align}\label{equation:EPPF}
p(\mathcal{C}) = V_n(t)\prod_{c\in\mathcal{C}} \gamma^{(|c|)}
\end{align}
where $t=|\mathcal{C}|$ is the number of parts in the partition, and
\begin{align}\label{equation:v}
 V_n(t) =\sum_{k = 1}^\infty\frac{k_{(t)}}{(\gamma k)^{(n)}}\,p_K(k).
\end{align}
\end{theorem}

All proofs have been collected in Appendix \ref{section:proofs}.  Here, $x^{(m)} =x(x+1)\cdots(x + m-1)$ and $x_{(m)} =x(x-1)\cdots(x - m+1)$, with $x^{(0)} = 1$ and $x_{(0)} = 1$ by convention.  We discuss computation of $V_n(t)$ in Section \ref{section:v}.  For comparison, under the DPM, the partition distribution induced by $Z_1,\ldots,Z_n$ is $p_{\scalebox{0.5}{\mbox{DPM}}}(\mathcal{C})=\frac{\alpha^t}{\alpha^{(n)}} \prod_{c\in\mathcal{C}} (|c|-1)!$ \citep{Antoniak_1974}.

Viewed as a function of the part sizes $(|c|:c\in\mathcal{C})$, Equation \ref{equation:EPPF} is an \emph{exchangeable partition probability function} (EPPF) in the terminology of \cite{Pitman_1995,Pitman_2006}, since it is a symmetric function of the part sizes. Consequently, $\mathcal{C}$ is an \emph{exchangeable random partition} of $[n]$; that is, its distribution is invariant under permutations of $[n]$ (alternatively, this can be seen directly from the definition of the model, since $Z_1,\dotsc,Z_n$ are exchangeable).

More specifically, we observe that Equation \ref{equation:EPPF} is a member of the family of Gibbs partition distributions \citep{Pitman_2006}; this is also implied by the results of \cite{Gnedin_2006} characterizing the extreme points of the space of Gibbs partition distributions. Further results on Gibbs partitions are provided by \citet{Ho_2007}, \citet{Lijoi_2008}, \citet{Cerquetti_2008,Cerquetti_2011}, \citet{Gnedin_2010}, and \citet{Lijoi_2010}. However, the utility of this representation for inference in mixture models with a prior on the number of components does not seem to have been previously explored in the literature.

Due to Theorem \ref{theorem:EPPF}, we have the following equivalent representation of the model:
\begin{align}
& \mathcal{C}\sim p(\mathcal{C}), \mbox{ with $p(\mathcal{C})$ as in Equation \ref{equation:EPPF}}\notag\\
& \phi_c\stackrel{\mathrm{iid}}{\sim} H  \mbox{ for $c\in\mathcal{C}$, given $\mathcal{C}$}\label{equation:model-C}\\
& X_j\sim f_{\phi_c} \mbox{ independently for $j \in c$, $c\in\mathcal{C}$, given $\phi,\mathcal{C}$,}\notag
\end{align}
where $\phi =(\phi_c: c\in\mathcal{C})$ is a tuple of $t =|\mathcal{C}|$ parameters $\phi_c\in\Theta$, one for each part $c\in\mathcal{C}$.

This representation is particularly useful for doing inference, since one does not have to deal with cluster labels or empty components. The formulation of models starting from a partition distribution has been a fruitful approach, exemplified by the development of product partition models \citep{Hartigan_1990,Barry_1992,Quintana_2003,Dahl_2009,Park_2010,Mueller_2010,Mueller_2011}.

\subsection{Basic properties}
\label{section:basic}

We list here some basic properties of the MFM model. See Appendix \ref{section:proofs} for proofs.
Denoting $x_c =(x_j: j\in c)$ and
$m(x_c) =\int_\Theta \big[\prod_{j\in c} f_\theta(x_j)\big]\,H(d\theta)$ (with the convention that $m(x_\emptyset) = 1$), we have
\begin{align}\label{equation:marginal-product}
p(x_{1:n}|\mathcal{C}) = \prod_{c\in\mathcal{C}} m(x_c).
\end{align}
The number of components $K$ and the number of clusters $T =|\mathcal{C}|$ are related by
\begin{align}
p(t|k) &= \frac{k_{(t)}}{(\gamma k)^{(n)}}\sum_{\mathcal{C}:|\mathcal{C}|= t}\prod_{c\in\mathcal{C}}\gamma^{(|c|)},\label{equation:ptk}\\
p(k|t) &= \frac{1}{V_n(t)}\frac{k_{(t)}}{(\gamma k)^{(n)}}\,p_K(k),\label{equation:pkt}
\end{align}
where in Equation \ref{equation:ptk}, the sum is over partitions $\mathcal{C}$ of $[n]$ such that $|\mathcal{C}|= t$.
The formula for $p(k|t)$ is required for doing inference about the number of components $K$ based on posterior samples of $\mathcal{C}$; fortunately, it is easy to compute.
We have the conditional independence relations
\begin{align}
\mathcal{C} & \perp K\mid T,\label{equation:CKT}\\
X_{1:n} & \perp K\mid T.\label{equation:XKT}
\end{align}

\subsection{The coefficients $V_n(t)$}
\label{section:v}

The following recursion is a special case of a more general result for Gibbs partitions \citep{Gnedin_2006}.

\begin{proposition}
The numbers $V_n(t)$ (Equation \ref{equation:v}) satisfy the recursion
\begin{align}\label{equation:recursion}
V_{n +1}(t+1) = V_n(t)/\gamma-(n/\gamma + t) V_{n +1}(t)
\end{align}
for any $0\leq t\leq n$ and $\gamma>0$.
\end{proposition}

This is easily seen by plugging the identity
$$k_{(t+1)} = (\gamma k + n)k_{(t)}/\gamma-(n/\gamma + t) k_{(t)} $$
into the expression for $V_{n+1}(t+1)$.
In the case of $\gamma = 1$, \cite{Gnedin_2010} has discovered a beautiful example of a distribution on $K$ for which both $p_K(k)$ and $V_n(t)$ have closed-form expressions.

In previous work on the MFM model, it has been common for $p_K$ to be chosen to be proportional to a Poisson distribution restricted to the positive integers or a subset thereof \citep{Phillips_1996,Stephens_2000,Nobile_2007}, and \cite{Nobile_2005} has proposed a theoretical justification for this choice.
Interestingly, the model has some nice mathematical properties if one instead chooses $K-1$ to be given a Poisson distribution, that is, $p_K(k) =\mathrm{Poisson}(k-1|\lambda)$ for some $\lambda>0$. One example of this arises here (for another example, see Section \ref{section:stick}): it turns out that if $p_K(k) =\mathrm{Poisson}(k-1|\lambda)$ and $\gamma= 1$ then
\begin{align}\label{equation:Poisson-V_n}
V_n(0) = \frac{1}{\lambda^n}\Big(1-\sum_{k=1}^n p_K(k)\Big).
\end{align}

However, to do inference, it is not necessary to choose $p_K$ to have any particular form. To do inference, we just need to be able to compute $p(\mathcal{C})$, and in turn, we need to be able to compute $V_n(t)$. To this end, note that $k_{(t)}/(\gamma k)^{(n)}\leq k^t/(\gamma k)^n$, and thus the infinite series for $V_n(t)$ converges rapidly when $t\ll n$. It always converges to a finite value when $1\leq t\leq n$; this is clear from the fact that $p(\mathcal{C})\in[0,1]$. This finiteness can also be seen directly from the series since $k^t/(\gamma k)^n \leq 1/\gamma^n$, and in fact, this shows that the series for $V_n(t)$ converges at least as rapidly (up to a constant) as the series $\sum_{k = 1}^\infty p_K(k)$ converges to $1$.
Hence, for any reasonable choice of $p_K$ (i.e., not having an extraordinarily heavy tail), $V_n(t)$ can easily be numerically approximated to a high level of precision. In practice, computing the required values of $V_n(t)$ takes a negligible amount of time.

\subsection{Self-consistent marginals}
\label{section:self-consistent}

For each $n = 1,2,\dotsc$, let $q_n(\mathcal{C})$ denote the distribution on partitions of $[n]$ as defined above (Equation \ref{equation:EPPF}).  This family of partition distributions is preserved under marginalization, in the following sense.

\begin{proposition}
\label{proposition:self-consistent}
If $m<n$ then $q_m$ coincides with the marginal distribution on partitions of $[m]$ induced by $q_n$.
\end{proposition}

In other words, drawing a sample from $q_n$ and removing elements $m+1,\dotsc,n$ from it yields a sample from $q_m$. This can be seen directly from the model definition (Equation \ref{equation:model}), since $\mathcal{C}$ is the partition induced by the $Z$'s, and the distribution of $Z_{1:m}$ is the same when the model is defined with any $n\geq m$. This property is sometimes referred to as \emph{consistency in distribution} \citep{Pitman_2006}.

By Kolmogorov's extension theorem (e.g., \citealp{Durrett_1996}), it is well-known that this implies the existence of a unique probability distribution on partitions of the positive integers $\mathbb{Z}_{>0}=\{1,2,\dotsc\}$ such that the marginal distribution on partitions of $[n]$ is $q_n$ for all $n\in\{1,2,\dotsc\}$.
A random partition of $\mathbb{Z}_{>0}$ from such a distribution is a \emph{combinatorial stochastic process}; for background, see \cite{Pitman_2006}.

\section{Restaurant process, stick-breaking, and random measure representations}
\label{section:representations}

\subsection{P\'{o}lya urn scheme / Restaurant process}
\label{section:restaurant}

\cite{Pitman_1996} considered a general class of urn schemes, or restaurant processes, corresponding to exchangeable partition probability functions (EPPFs). The following scheme for the MFM falls into this general class.

\begin{theorem}
\label{theorem:restaurant}
The following process generates partitions $\mathcal{C}_1,\mathcal{C}_2,\dotsc$ such that for any $n\in\{1,2,\dotsc\}$, the probability mass function of $\mathcal{C}_n$ is given by Equation \ref{equation:EPPF}.
\begin{itemize}
\item Initialize with a single cluster consisting of element $1$ alone: $\mathcal{C}_1 =\{\{1\}\}$.
\item For $n = 2,3,\dotsc$, element $n$ is placed in \dots
\begin{itemize}
\item[] an existing cluster $c\in\mathcal{C}_{n-1}$ with probability $\propto |c|+\gamma $
\item[] a new cluster with probability $\displaystyle\propto \frac{V_n(t+1)}{V_n(t)}\gamma$
\end{itemize}
where $t =|\mathcal{C}_{n-1}|$.
\end{itemize}
\end{theorem}

Clearly, this bears a close resemblance to the Chinese restaurant process (i.e., the Blackwell--MacQueen urn process), in which the $n$th element is placed in an existing cluster $c$ with probability $\propto |c|$ or a new cluster with probability $\propto \alpha$ (the concentration parameter) \citep{Blackwell_1973,aldous1985exchangeability}.

\subsection{Random discrete measures}
\label{section:discrete-measure}

The MFM can also be formulated starting from a distribution on discrete measures that is analogous to the Dirichlet process. With $K$, $\pi$, and $\theta_{1:K}$ as in Equation \ref{equation:model}, let
$$ G =\sum_{i = 1}^K \pi_i \delta_{\theta_i} $$
where $\delta_\theta$ is the unit point mass at $\theta$. Let us denote the distribution of $G$ by $\mathcal{M}(p_K,\gamma,H)$. Note that $G$ is a random discrete measure over $\Theta$. If $H$ is continuous (i.e., $H(\{\theta\}) = 0$ for all $\theta\in\Theta$), then with probability $1$, the number of atoms is $K$; otherwise, there may be fewer than $K$ atoms.
If we take $X_1,\dotsc,X_n|G$ i.i.d.\ from the resulting mixture, namely,
$$f_G(x):=\int f_\theta(x)G(d\theta) = \sum_{i = 1}^K \pi_i f_{\theta_i}(x), $$
then the distribution of $X_{1:n}$ is the same as before.
So, in this notation, the MFM model is:
\begin{align*}
& G\sim\mathcal{M}(p_K,\gamma,H)\\
& X_1,\dotsc,X_n\stackrel{\mathrm{iid}}{\sim} f_G, \mbox{ given $G$.}
\end{align*}

This random discrete measure perspective is connected to work on \emph{species sampling models} \citep{Pitman_1996} in the following way. When $H$ is continuous, we can construct a species sampling model by letting $G\sim\mathcal{M}(p_K,\gamma,H)$ and modeling the observed data as $\beta_1,\dotsc,\beta_n\sim G$. We refer to \cite{Pitman_1996}, \cite{Hansen_2000}, \cite{Ishwaran_2003}, \cite{Lijoi_2005b,Lijoi_2007}, and \cite{Lijoi_2008} for more background on species sampling models and further examples.
The following posterior predictive rule for this particular model follows the form of Pitman's general rule; note the close relationship to the restaurant process (Theorem \ref{theorem:restaurant} above).

\begin{theorem}
\label{theorem:species-predictive}
If $H$ is continuous, then $\beta_1\sim H$ and the distribution of $\beta_n$ given $\beta_1,\dotsc,\beta_{n-1}$ is proportional to
\begin{align}\label{equation:species-predictive}
\frac{V_n(t+1)}{V_n(t)}\gamma H + \sum_{i = 1}^t (n_i+\gamma)\delta_{\beta_i^*},
\end{align}
where $\beta^*_1,\dotsc,\beta^*_t$ are the distinct values taken by $\beta_1,\dotsc,\beta_{n-1}$, and $n_i =\#\big\{j\in[n-1]:\beta_j =\beta_i^*\big\}$.
\end{theorem}

For comparison, when $G\sim\mathrm{DP}(\alpha H)$ instead, the distribution of $\beta_n$ given $\beta_1,\dotsc,\beta_{n-1}$ is proportional to $\alpha H + \sum_{j = 1}^{n-1} \delta_{\beta_j} = \alpha H +\sum_{i = 1}^t n_i\beta_i^*$, since $G|\beta_1,\dotsc,\beta_{n-1}\sim\mathrm{DP}(\alpha H +\sum_{j = 1}^{n-1}\delta_{\beta_j})$ \citep{Ferguson_1973,Blackwell_1973}.

\subsection{Stick-breaking representation}
\label{section:stick}

The Dirichlet process has an elegant stick-breaking representation for the mixture weights $\pi_1,\pi_2,\dotsc$ \citep{Sethuraman_1994,Sethuraman_1981}. This extraordinarily clarifying perspective has inspired a number of  nonparametric models \citep{MacEachern_1999,MacEachern_2000,Hjort_2000,Ishwaran_2000,Ishwaran_2001b,Griffin_2006,Dunson_2008,Chung_2009,Rodriguez_2011,Broderick_2012}, has provided insight into the properties of related models \citep{Favaro_2012,Teh_2007,Thibaux_2007,Paisley_2010}, and has been used to develop efficient inference algorithms \citep{Ishwaran_2001b,Blei_2006,Papaspiliopoulos_2008,Walker_2007b,Kalli_2011}.

In a certain special case---namely, when $p_K(k) =\mathrm{Poisson}(k-1|\lambda)$ and $\gamma = 1$---we have noticed that the MFM also has an interesting representation that we describe using the stick-breaking analogy, although it is somewhat different in nature. This is another example of the nice mathematical properties resulting from this choice of $p_K$ and $\gamma$.
Consider the following procedure:
\begin{quote}
\emph{Take a unit-length stick, and break off i.i.d.\ $\mathrm{Exponential}(\lambda)$ pieces until you run out of stick.}
\end{quote}
In other words, let $\epsilon_1,\epsilon_2,\dotsc\stackrel{\mathrm{iid}}{\sim}\mathrm{Exponential}(\lambda)$, define $\tilde K =\min\{j:\sum_{i = 1}^j\epsilon_i \geq 1\}$, and set $\tilde\pi_i =\epsilon_i$ for $i = 1,\dotsc,\tilde K - 1$ and $\tilde\pi_{\tilde K} = 1-\sum_{i = 1}^{\tilde K-1}\tilde\pi_i$.

\begin{proposition}
\label{proposition:stick}
The stick lengths $\tilde\pi$ have the same distribution as the mixture weights $\pi$ in the MFM model when $p_K(k) =\mathrm{Poisson}(k-1|\lambda)$ and $\gamma = 1$.
\end{proposition}

This is a consequence of a standard construction for Poisson processes. This suggests a way of generalizing the MFM model: take any sequence of nonnegative random variables $(\epsilon_1,\epsilon_2,\dotsc)$ (not necessarily independent or identically distributed) such that $\sum_{i = 1}^\infty\epsilon_i>1$ with probability $1$, and define $\tilde K$ and $\tilde\pi$ as above. Although the distribution of $\tilde K$ and $\tilde\pi$ may be complicated, in some cases it might still be possible to do inference based on the stick-breaking representation. This might be an interesting way to introduce different kinds of prior information on the mixture weights, however, we have not explored this possibility.

\section{Asymptotics}
\label{section:asymptotics}

In this section, we consider the asymptotics of $V_n(t)$, the asymptotic relationship between the number of components and the number of clusters, and the approximate form of the conditional distribution on cluster sizes given the number of clusters.

\subsection{Asymptotics of $V_n(t)$}
\label{section:v-asymptotics}

Recall that $V_n(t) =\sum_{k = 1}^\infty\frac{k_{(t)}}{(\gamma k)^{(n)}}\,p_K(k)$ (Equation \ref{equation:v})
for $1\leq t\leq n$, with $\gamma>0$ and $p_K$ a p.m.f.\ on $\{1,2,\dotsc\}$.

\begin{theorem}
\label{theorem:v-asymptotics}
For any $t\in\{1,2,\dotsc\}$, if $p_K(t)>0$ then
\begin{align}
V_n(t)\sim\frac{t_{(t)}}{(\gamma t)^{(n)}}\,p_K(t) \sim \frac{t!}{n!}\,\frac{\Gamma(\gamma t)}{n^{\gamma t - 1}}\,p_K(t)
\end{align}
as $n\to\infty$.
\end{theorem}

In particular, $V_n(t)$ has a simple interpretation, asymptotically---it behaves like the $k = t$ term in the series.

\subsection{Relationship between the number of clusters and number of components}
\label{section:clusters-components-relationship}

In the MFM, it is perhaps intuitively clear that, under the prior at least, the number of clusters $T =|\mathcal{C}|$ behaves very similarly to the number of components $K$ when $n$ is large. It turns out that under the posterior they also behave very similarly for large $n$.

\begin{theorem}
\label{theorem:clusters-components-relationship}
Let $x_1,x_2,\dotsc\in\mathcal{X}$ and $k\in\{1,2,\dotsc\}$. If $p_K(1),\dotsc,p_K(k)>0$ then
$$ \big|p(T = k\mid x_{1:n}) - p(K = k\mid x_{1:n})\big| \longrightarrow 0 $$
as $n\to\infty$.
\end{theorem}

\subsection{Distribution of the cluster sizes under the prior}
\label{section:part-sizes}

Here, we examine one of the major differences between the MFM and DPM priors. Roughly speaking, under the prior, the MFM prefers all clusters to be the same order of magnitude, while the DPM prefers having a few large clusters and many very small clusters.  In the following calculations, we quantify the preceding statement more precisely. (See \cite{Green_2001} for informal observations along these lines.) Interestingly, these prior influences remain visible in certain aspects of the posterior, even in the limit as $n$ goes to infinity, as shown by the inconsistency of DPMs for the number of components in a finite mixture \citep{Miller_2014}.

Let $\mathcal{C}$ be the partition of $[n]$ in the MFM model (Equation \ref{equation:EPPF}), and let $A =(A_1,\dotsc,A_T)$ be the ordered partition of $[n]$ obtained by randomly ordering the parts of $\mathcal{C}$, uniformly among the $T!$ possible choices, where $T =|\mathcal{C}|$. Then
$$ p(A) = \frac{p(\mathcal{C})}{|\mathcal{C}|!} =\frac{1}{t!} V_n(t)\prod_{i = 1}^t\gamma^{(|A_i|)},$$
where $t =|\mathcal{C}|$.
Now, let $S =(S_1,\dotsc,S_T)$ be the vector of part sizes of $A$, that is, $S_i =|A_i|$. Then
$$ p(S = s) =\sum_{A: S(A) = s} p(A) = V_n(t)\,\frac{n!}{t!}\prod_{i = 1}^t\frac{\gamma^{(s_i)}}{s_i!} $$
for $s\in\Delta_t$, $t\in\{1,\ldots,n\}$, where $\Delta_t =\{s\in\mathbb{Z}^t:\sum_i s_i = n,\,s_i\geq 1\,\forall i\}$ (i.e., the $t$-part compositions of $n$).
For any $x>0$, writing $x^{(m)}/m! =\Gamma(x + m)/(m!\,\Gamma(x))$ and using Stirling's approximation, we have
$x^{(m)} / m! \sim m^{x - 1} / \Gamma(x)$ as $m\to\infty$. This yields the approximations
$$ p(S = s) \approx \frac{V_n(t)}{\Gamma(\gamma)^t}\,\frac{n!}{t!}\,\prod_{i = 1}^t s_i^{\gamma - 1}
\approx \frac{p_K(t)}{n^{\gamma t-1}}\,\frac{\Gamma(\gamma t)}{\Gamma(\gamma)^t}\,\prod_{i = 1}^t s_i^{\gamma - 1}$$
(using Theorem \ref{theorem:v-asymptotics} in the second step), and
$$ p(S = s\mid T = t)\approx \kappa\prod_{i = 1}^t s_i^{\gamma - 1} $$
for $s\in\Delta_t$, where $\kappa$ is a normalization constant. Thus $p(s|t)$, although a discrete distribution, has approximately the same shape as a symmetric $t$-dimensional Dirichlet distribution. This would be obvious if we were conditioning on the number of components $K$, and it makes intuitive sense when conditioning on $T$, since $K$ and $T$ are essentially the same for large $n$.

It is very interesting to compare this to the corresponding distributions for Dirichlet process mixtures. In the  DPM, we have $p_{\scalebox{0.5}{\mbox{DPM}}}(\mathcal{C}) =\frac{\alpha^t}{\alpha^{(n)}}\prod_{c\in\mathcal{C}}(|c|- 1)!$, and $p_{\scalebox{0.5}{\mbox{DPM}}}(A) = p_{\scalebox{0.5}{\mbox{DPM}}}(\mathcal{C})/|\mathcal{C}|!$ as before, so for $s\in\Delta_t$, $t\in\{1,\ldots,n\}$,
$$ p_{\scalebox{0.5}{\mbox{DPM}}}(S =s) =  \frac{n!}{\alpha^{(n)}}\,\frac{\alpha^t}{t!}\,s_1^{-1}\cdots s_t^{-1}$$
and
$$ p_{\scalebox{0.5}{\mbox{DPM}}}(S = s\mid T = t)\propto s_1^{-1}\cdots s_t^{-1}, $$
which has the same shape as a $t$-dimensional Dirichlet distribution with all the parameters taken to $0$ (noting that this is normalizable since $\Delta_t$ is finite). Asymptotically in $n$, $p_{\scalebox{0.5}{\mbox{DPM}}}(s|t)$ puts all of its mass in the ``corners'' of the discrete simplex $\Delta_t$, while under the MFM, $p(s|t)$ remains more evenly dispersed.

\section{Inference algorithms}
\label{section:inference}

As shown by the results of Sections \ref{section:partitions} and \ref{section:representations}, MFMs have many of the same properties as DPMs. As a result, much of the extensive body of work on MCMC samplers for DPMs can be directly applied to MFMs, including samplers for conjugate and non-conjugate cases, as well as split-merge samplers.

When $H$ is a conjugate prior for $\{f_\theta\}$, such that the marginal likelihood $m(x_c) =\int_\Theta \big[\prod_{j\in c} f_\theta(x_j)\big]\,H(d\theta)$ can be easily computed, the following Gibbs sampling algorithm can be used to sample from the posterior on partitions, $p(\mathcal{C}|x_{1:n})$. Given a partition $\mathcal{C}$, let $\mathcal{C}\setminus j$ denote the partition obtained by removing element $j$ from $\mathcal{C}$.
\begin{enumerate}
\item Initialize $\mathcal{C} =\{[n]\}$ (i.e., one cluster).
\item Repeat the following $N$ times, to obtain $N$ samples.
\begin{itemize}
\item[] For $j = 1,\dotsc,n$:  Remove element $j$ from $\mathcal{C}$, and place it \dots
\begin{itemize}
\item[] in $c\in\mathcal{C}\setminus j$ with probability $\displaystyle\propto (|c|+\gamma)\frac{m(x_{c\,\cup j})}{m(x_c)}$
\item[] in a new cluster with probability $\displaystyle\propto \gamma\, \frac{V_n(t+1)}{V_n(t)}\,m(x_j)$
\end{itemize}
where $t=|\mathcal{C}\setminus j|$.
\end{itemize}
\end{enumerate}
This is a direct adaptation of ``Algorithm 3'' for DPMs \citep{MacEachern_1994,Neal_1992,Neal_2000}. The only differences are that in Algorithm 3, $|c|+\gamma$ is replaced by $|c|$, and $\gamma V_n(t+1)/V_n(t)$ is replaced by $\alpha$ (the concentration parameter). Thus, the differences between the MFM and DPM versions of the algorithm are precisely the same as the differences between their respective restaurant processes.
Computing the required values of $V_n(t)$ takes a negligible amount of time compared to running the sampler.
In order for the algorithm to be valid, the Markov chain needs to be irreducible, and to achieve this it is necessary to have $\big\{t\in\{1,2,\dotsc\}:V_n(t)>0\big\}$ be a block of consecutive integers. In fact, it turns out that this is always the case (and this block includes $t = 1$), since for any $k$ such that $p_K(k)>0$, we have $V_n(t)>0$ for all $t = 1,\dotsc,k$.

When $H$ is a non-conjugate prior (and $m(x_c)$ cannot be easily computed), a clever auxiliary variable technique referred to as ``Algorithm 8'' can be used for inference in the DPM \citep{Neal_2000,MacEachern_1998}. Making the same substitutions as above, we can apply Algorithm 8 to perform inference in the MFM as well; see \citet{Miller_thesis} for details.

A well-known issue with incremental Gibbs samplers such as these, however, when applied to DPMs, is that the mixing can be somewhat slow, since it may take a long time to create or destroy substantial clusters by moving one element at a time. With MFMs, this issue seems to be exacerbated, since MFMs tend to put small probability (compared with DPMs) on partitions with tiny clusters (see Section \ref{section:part-sizes}), making it difficult for the sampler to move through these regions of the space.

To deal with this issue, split-merge samplers for DPMs have been developed, in which a large number of elements can be reassigned in a single move \citep{Dahl_2003,Dahl_2005,Jain_2004,Jain_2007}. In the same way as the incremental samplers, one can directly apply these split-merge samplers (both conjugate and non-conjugate) to MFMs, using the properties described in Sections \ref{section:partitions} and \ref{section:representations}. More generally, it seems likely that any partition-based MCMC sampler for DPMs could be applied to MFMs as well.

In Section \ref{section:empirical}, we apply the Jain--Neal split-merge samplers coupled with incremental Gibbs samplers, in both conjugate and non-conjugate settings.

\section{Empirical demonstrations}
\label{section:empirical}

In this section, we demonstrate the MFM on simulated and real datasets. All of the examples below involve Gaussian component densities, but of course our approach is not limited to mixtures of Gaussians.

\subsection{Simulation example}
\label{section:simulation}

In the introduction, we presented several figures comparing the MFM and DPM on data from a three-component bivariate Gaussian mixture, illustrating the behavior of the MFM with respect to density estimation, clustering, and inference for the number of components. Here, we provide the details of the data, model, and method of inference for this simulation example.

\subsubsection*{Data}

The data distribution is
$\sum_{i = 1}^3 w_i \mathcal{N}(\mu_i,C_i)$ where $w =(0.45,0.3,0.25)$, $\mu_1 =\matrixsmall{4\\4}$, $\mu_2 =\matrixsmall{7\\4}$, $\mu_3 =\matrixsmall{6\\2}$,
$C_1 = \matrixsmall{1 & 0\\ 0 & 1}$,
$C_2 = R\matrixsmall{2.5 & 0\\ 0 & 0.2}R^\mathtt{T}$ where
$R =\matrixsmall{\cos\rho & \, -\sin\rho \\ \sin\rho & \cos\rho}$ with $\rho=\pi/4$, and
$C_3 = \matrixsmall{3 & 0\\ 0 & 0.1}$.

\subsubsection*{Model}

The component densities are multivariate normal,
$f_\theta(x) = f_{\mu,\Lambda}(x) =\mathcal{N}(x|\mu,\Lambda^{-1})$
and the base measure (prior) $H$ on $\theta =(\mu,\Lambda)$ is
$\mu\sim\mathcal{N}(\widehat\mu,\widehat C)$, $\Lambda\sim\mathrm{Wishart}_d(V,\nu)$
independently, where $\widehat\mu$ is the sample mean, $\widehat C$ is the sample covariance, $\nu=d=2$, and $V=\widehat C^{-1}/\nu$.  Here, $\mathrm{Wishart}_d(\Lambda|V,\nu)\propto |\det \Lambda|^{(\nu-d-1)/2} \exp\big(-\tfrac{1}{2} \mathrm{tr}(V^{-1}\Lambda)\big)$. Note that this is a data-dependent prior.

For the MFM, we take $K\sim\mathrm{Geometric}(r)$ ($p_K(k) = (1-r)^{k-1} r$ for $k = 1,2,\dotsc$) with $r = 0.1$, and we choose $\gamma = 1$ for the finite-dimensional Dirichlet parameters.
For the DPM, we put an $\mathrm{Exponential}(1)$ prior on the concentration parameter, $\alpha$.

Note that taking $\mu$ and $\Lambda$ to be independent results in a non-conjugate prior. This prior is appropriate when the location of the data is not informative about the covariance (and vice versa).

\subsubsection*{Inference}

For both the MFM and DPM, we use the non-conjugate split-merge sampler of \citet{Jain_2007}, coupled with Algorithm 8 of \citet{Neal_2000} (using a single auxiliary variable) for incremental Gibbs updates to the partition. Specifically, following \citet{Jain_2007}, we use the (5,1,1,5) scheme: 5 intermediate scans to reach the split launch state, 1 split-merge move per iteration, 1 incremental Gibbs scan per iteration, and 5 intermediate moves to reach the merge launch state. Gibbs updates to the DPM concentration parameter $\alpha$ are made using Metropolis--Hastings moves.

Five independent datasets were used for each $n \in\{50,100,250,1000\}$, and for each model (MFM and DPM), the sampler was run for 5,000 burn-in iterations and 95,000 sample iterations (for a total of 100,000). Judging by traceplots and running averages of various statistics, this appeared to be sufficient for mixing. The cluster sizes were recorded after each iteration, and to reduce memory storage requirements, the full state of the chain was recorded only once every 100 iterations. For each run, the seed of the random number generator was initialized to the same value for both the MFM and DPM.

For a dataset of size $n$, the sampler used for these experiments took approximately $8\times 10^{-6}\,n$ seconds per iteration, using a 2.80 GHz processor with 6 GB of RAM.

\subsubsection*{Results}

As described in the introduction, the results of this simulation empirically indicate that on data from a finite mixture, MFMs and DPMs are consistent for the density (Figures \ref{figure:density-estimates} and \ref{figure:Hellinger}), DPM clusterings tend to have small extra clusters while MFM clusterings do not (Figure \ref{figure:clustering}), and MFMs are consistent for the number of components while DPMs are not (Figure \ref{figure:tk-posteriors}).  This is what we expect from theory (although to be precise, the inconsistency result of \citet{Miller_2014} only applies to the case of fixed concentration parameter $\alpha$).
These results are not too surprising, since when the data distribution is a finite mixture from the assumed family, the MFM is correctly specified, while the DPM is not.  On data from an infinite mixture, one would expect the DPM to have certain advantages.
See Appendix \ref{section:formulas} for formulas for computing the posterior on $k$ and the density estimates.

\subsection{Galaxy dataset}
\label{section:galaxy}

The galaxy dataset \citep{Roeder_1990} is a standard benchmark for mixture models, consisting of measurements of the velocities of $82$ galaxies in the Corona Borealis region; see Figure \ref{figure:galaxy}.
The purpose of this example is to demonstrate agreement between our method and published results using reversible jump MCMC with the same model, and also to show that using hyperpriors presents no difficulties.

\subsubsection*{Model}

To enable comparison, we use exactly the same model as \citet{Richardson_1997}.
The component densities are univariate normal,
$f_\theta(x) = f_{\mu,\lambda}(x) =\mathcal{N}(x|\mu,\lambda^{-1})$,
and the base measure $H$ on $\theta =(\mu,\lambda)$ is
$\mu\sim\mathcal{N}(\mu_0,\sigma_0^2)$, $\lambda\sim\mathrm{Gamma}(a,b)$
independently (where $\mathrm{Gamma}(\lambda|a,b)\propto \lambda^{a-1} e^{-b \lambda}$). Further, a hyperprior is placed on $b$, by taking $b\sim\mathrm{Gamma}(a_0,b_0)$. The remaining parameters are set to
$\mu_0 =(\max\{x_i\}+\min\{x_i\})/2$, $\sigma_0 =\max\{x_i\}-\min\{x_i\}$, $a = 2$, $a_0 = 0.2$, and $b_0 = 10/\sigma_0^2$. Note that the parameters $\mu_0$, $\sigma_0$, and $b_0$ are functions of the observed data $x_1,\dotsc,x_n$.
See \citet{Richardson_1997} for the rationale behind these parameter choices. (Note: This choice of $\sigma_0$ may be a bit too large, affecting the posteriors on the number of clusters and components, however, we stick with it to enable comparisons to \citet{Richardson_1997}.)
For the MFM, following \citet{Richardson_1997}, we take $K\sim\mathrm{Uniform}\{1,\allowbreak\dotsc,30\}$ and $\gamma = 1$. For the DPM, we take $\alpha\sim\mathrm{Exponential}(1)$.

\subsubsection*{Inference}

As before, we use the non-conjugate split-merge sampler of \citet{Jain_2007} coupled with Algorithm 8 of \citet{Neal_2000}, and Gibbs updates to the DPM concentration parameter $\alpha$ are made using Metropolis--Hastings.
We use Gibbs sampling to handle the hyperprior on $b$ (i.e., append $b$ to the state of the Markov chain, run the sampler given $b$ as usual, and periodically sample $b$ given everything else). More general hyperprior structures can be handled similarly. In all other respects, the same inference algorithm as in Section \ref{section:simulation} was used.

We do not restrict the parameter space in any way (e.g., forcing the component means to be ordered to obtain identifiability, as was done by \citet{Richardson_1997}). All of the quantities we consider are invariant to the labeling of the clusters. See \citet{Jasra_2005} for discussion on this point.

The sampler was run for 5,000 burn-in iterations, and 45,000 sample iterations.  This appeared to be more than sufficient for mixing. Cluster sizes were recorded after each iteration, and the full state of the chain was recorded every 50 iterations.
Each iteration took approximately $8\times 10^{-6}\,n$ seconds, with $n = 82$.

\subsubsection*{Results}

Figure \ref{figure:galaxy} shows the estimated densities and the posteriors on the number of clusters and components.  Comparing this with Figure 2(c) of \citet{Richardson_1997}, we see that our MFM density estimate is visually indistinguishable from theirs (as it should be, since we are using the same model with the same parameters).

\begin{figure}
\centering
\includegraphics[trim=0.8cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{galaxy-densities.png}
\includegraphics[trim=0.8cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{galaxy-tk_posteriors.png}
\caption{Results on the galaxy dataset. Left: Histogram of the data (green bars), rug plot of the data (black ticks), and estimated densities using the MFM (red solid line) and DPM (blue dashed line). Right: MFM and DPM posteriors on the number of clusters ($t$), along with the MFM posterior on the number of components ($k$).}
\label{figure:galaxy}
\end{figure}

Table \ref{table:galaxy} compares our estimate of the MFM posterior on the number of components $k$ with the results of \citet{Richardson_1997}. Again, the results are very close, as expected.

\begin{table}[!ht]
\centering
\caption{Estimate of the MFM posterior on $k$ for the galaxy dataset.}%
\label{table:galaxy}%
\begin{tabular}{|c|c|c|c|c|c|c|c|}%
\hline
$k$ & 1 & 2 & 3 & 4 & 5 & 6 & 7 \\
\hline
Here & 0.000 & 0.000 & 0.065 & 0.143 & 0.191 & 0.191 & 0.153 \\
R\&G & 0.000 & 0.000 & 0.061 & 0.128 & 0.182 & 0.199 & 0.160 \\
\hline
\end{tabular}\\[1em]
\begin{tabular}{|c|c|c|c|c|c|c|c|}%
\hline
8 & 9 & 10 & 11 & 12 & 13 & 14 & 15 \\
\hline
0.106 & 0.066 & 0.039 & 0.021 & 0.012 & 0.006 & 0.003 & 0.002 \\
0.109 & 0.071 & 0.040 & 0.023 & 0.013 & 0.006 & 0.003 & 0.002 \\
\hline
\end{tabular}
\end{table}

\subsection{Discriminating cancer types using gene expression data}
\label{section:gene-expression}

In cancer research, gene expression profiling---that is, measuring the degree to which each gene is expressed by a given tissue sample under given conditions---enables the identification of distinct subtypes of cancer, leading to greater understanding of the mechanisms underlying cancers as well as potentially providing patient-specific diagnostic tools.  In gene expression datasets, there are typically a small number of very high-dimensional data points, each consisting of the gene expression levels in a given tissue sample under given conditions.

One approach to analyzing gene expression data is to use Gaussian mixture models to identify clusters which may represent distinct cancer subtypes \citep{Yeung_2001,mclachlan2002mixture,Medvedovic_2002,Medvedovic_2004,deSouto_2008,Rasmussen_2009,McNicholas_2010}. In fact, in a comparative study of seven clustering methods on 35 cancer gene expression datasets with known ground truth, \citet{deSouto_2008} found that finite mixtures of Gaussians provided the best results---when the number of components $k$ was set to the true value.  However, in practice, choosing an appropriate value of $k$ can be difficult.  Using the methods developed in this paper, the MFM provides a principled approach to inferring the clusters even when $k$ is unknown, as well as doing inference for $k$, provided that the components are well-modeled by Gaussians. (However, see Section \ref{section:discussion} for some potential pitfalls.)

The purpose of this example is to demonstrate that our approach can work well even in very high-dimensional settings, and may provide a useful tool for this application. It should be emphasized that we are not cancer scientists, so the results reported here should not be interpreted as scientifically relevant, but simply as a proof-of-concept.

\subsubsection*{Data}

We apply the MFM to gene expression data collected by \citet{Armstrong_2001} in a study of leukemia subtypes. \citet{Armstrong_2001} measured gene expression levels in samples from 72 patients who were known to have one of two leukemia types, acute lymphoblastic leukemia (ALL) or acute myelogenous leukemia (AML), and they found that a previously undistinguished subtype of ALL, which they termed mixed-lineage leukemia (MLL), could be distinguished from conventional ALL and AML based on the gene expression profiles.

We use the preprocessed data provided by \citet{deSouto_2008}, which they filtered to include only genes with expression levels differing by at least 3-fold in at least 30 samples, relative to their mean expression level across all samples.  The resulting dataset consists of 72 samples and 1081 genes per sample, i.e., $n = 72$ and $d = 1081$. Following standard practice, we take the base-2 logarithm of the data before analysis, and normalize each dimension to have zero mean and unit variance.

\subsubsection*{Model}

For simplicity, we use multivariate Gaussian component densities with diagonal covariance matrices, i.e., the dimensions are independent univariate Gaussians, and we place independent conjugate priors on each dimension.  Thus, for each component, for $i = 1,\dotsc,d$, dimension $i$ is $\mathcal{N}(\mu_i,\lambda_i^{-1})$,
with $\lambda_i\sim\mathrm{Gamma}(a,b)$ and $\mu_i|\lambda_i\sim\mathcal{N}(0,(c\lambda_i)^{-1})$.
We choose $a=1$, $b=1$, and $c=1$. (Recall that the data is zero mean, unit variance in each dimension.)
For the MFM, $K\sim\mathrm{Geometric}(0.1)$ and $\gamma = 1$, and for the DPM, $\alpha\sim\mathrm{Exponential}(1)$.
These are all simply default settings and have not been tailored to the problem; a careful scientific investigation would involve thorough prior elicitation, sensitivity analysis, and model checking.

\subsubsection*{Inference}

Given the partition $\mathcal{C}$ of the data into clusters, the parameters can be integrated out analytically since the prior is conjugate. Thus, for both the MFM and DPM, we use the split-merge sampler of \citet{Jain_2004} for conjugate priors, coupled with Algorithm 3 of \citet{Neal_2000}. Following \citet{Jain_2004}, we use the (5,1,1) scheme: 5 intermediate scans to reach the split launch state, 1 split-merge move per iteration, and 1 incremental Gibbs scan per iteration.

Due to the high-dimensionality of the parameters, this has far better mixing time than sampling the parameters, as is done in reversible jump MCMC.

The sampler was run for 1,000 burn-in iterations, and 19,000 sample iterations.  This appears to be many more iterations than required for burn-in and mixing in this particular example---in fact, only 5 to 10 iterations are required to separate the clusters, and the results are indistinguishable when using only 10 burn-in and 190 sample iterations.
The full state of the chain was recorded every 20 iterations.
Each iteration took approximately $1.3\times 10^{-3}\,n$ seconds, with $n = 72$.

\subsubsection*{Results}

The posteriors on the number of clusters $t$ are concentrated at $3$ (see Figure \ref{figure:gene}), in agreement with the division into ALL, MLL, and AML determined by \citet{Armstrong_2001}. The MFM posterior on $k$ is shifted slightly to the right because there are a small number of observations; this accounts for uncertainty regarding the possibility of additional components that were not observed in the given data.

\begin{figure}
\centering
\includegraphics[trim=0.8cm 0.5cm 1cm 0.2cm, clip=true, width=0.49\textwidth]{tk_posteriors_armstrong-2002-v1.png}
\includegraphics[trim=7cm 3cm 5cm 0cm, clip=true, width=0.49\textwidth]{pairwise-armstrong.png}
\caption{Results on the leukemia gene expression dataset. Left: Posteriors on the number of clusters and components. Right: MFM pairwise probability matrix (the DPM matrix is the same).  See the text for discussion.}
\label{figure:gene}
\end{figure}

Figure \ref{figure:gene} also shows the MFM pairwise probability matrix, that is, the matrix in which entry $(i,j)$ is the posterior probability that data points $i$ and $j$ belong to the same cluster; in the figure, white is probability 0, black is probability 1. (The DPM matrix, not shown, is indistinguishable from the MFM matrix.)
The rows and columns of the matrix are ordered according to ground truth, such that 1--24 are ALL, 25--44 are MLL, and 45--72 are AML.
The model has clearly separated the subjects into these three groups, with a small number of exceptions: subject 41 is clustered with the AML subjects instead of MLL, subject 52 with the MLL subjects instead of AML, and subject 17 is about 50\% ALL and 50\% MLL.

\section{Discussion}
\label{section:discussion}

Due to the fact that inference for the number of components is a topic of high interest in many research communities, it seems prudent to make some cautionary remarks in this regard.
Many approaches have been proposed for estimating the number of components \citep{Henna_1985,Keribin_2000,Leroux_1992,Ishwaran_2001,James_2001,Henna_2005,Woo_2006,Woo_2007}. In theory, the MFM model provides a Bayesian approach to consistently estimating the number of components, making it a potentially attractive method of assessing the heterogeneity of the data.  However, there are several possible pitfalls to consider, some of which are more obvious than others.

An obvious potential issue is that in many applications, the clusters which one wishes to distinguish are purely notional (for example, perhaps, clusters of images or documents), and a mixture model is used for practical purposes, rather than because the data is actually thought to arise from a mixture. Clearly, in such cases, inference for the ``true'' number of components is meaningless. On the other hand, in some applications, the data definitely comes from a mixture (for example, extracellular recordings of multiple neurons)---so there is in reality a true number of components---however, usually the form of the mixture components is far from clear.

More subtle issues are that the posteriors on $k$ and $t$ can be
\begin{enumerate}
\item strongly affected by the base measure $H$, and
\item sensitive to misspecification of the family of component distributions $\{f_\theta\}$.
\end{enumerate}
Issue (1) can be seen, for instance, in the case of normal mixtures: it might seem desirable to choose the prior on the component means to have large variance in order to be less informative, however, this causes the posteriors on $k$ and $t$ to favor smaller values \citep{Richardson_1997,Stephens_2000,Jasra_2005}. The basic mechanism at play here is the same as in the Bartlett--Lindley paradox, and shows up in many Bayesian model selection problems.
With some care, this issue can be dealt with by varying the base measure $H$ and observing the effect on the posterior---that is, by performing a sensitivity analysis---for instance, see \cite{Richardson_1997}.

Issue (2) is more serious---in practice, we typically cannot expect our choice of $\{f_\theta:\theta\in\Theta\}$ to contain the true component densities (assuming the data is even from a mixture). When the model is misspecified in this way, the posteriors of $k$ and $t$ can be severely affected and depend strongly on $n$. For instance, if the model uses mixtures of Gaussians, and the true data distribution is not a finite mixture of Gaussians, then these posteriors can be expected to diverge to infinity as $n$ increases. Consequently, the effects of misspecification need to be carefully considered if these posteriors are to be used as measures of heterogeneity.  Steps toward addressing the issue of robustness have been taken by \cite{Woo_2006,Woo_2007} and \cite{rodriguez2014univariate}, however, this is an important problem demanding further study.

Despite these issues, sample clusterings and estimates of the number of components or clusters can provide a useful tool for exploring complex datasets, particularly in the case of high-dimensional data that cannot easily be visualized. It should always be borne in mind, though, that the results can be interpreted as being correct only to the extent that the model assumptions are correct.

\section*{Acknowledgments}

We are very grateful to Steve MacEachern for many helpful suggestions.
This work was supported in part by the National Science Foundation (NSF) grants DMS-1007593 and DMS-1309004, by the National Institute of Mental Health (NIMH) grant R01MH102840, and by the Defense Advanced Research Projects Agency (DARPA) contract FA8650-11-1-715.

redpajama_set_name: RedPajamaArXiv


text:
Q: Split card based on overflow of data I wish to achieve a UI like this, where when the data overflows, it continues on the next card. Except I will have fixed height cards as compared to this below picture.
I would like to hear the possible ways to achieve this, also I'm making this in a react native project.
P.S: This is not the use case where I can do it using numColumns prop of FlatList.
As data here is not from 2 different data set (in Array), but from a single data set (in Array)

redpajama_set_name: RedPajamaStackExchange


text:
PDH steht für:
 Plesiochrone Digitale Hierarchie, eine international standardisierte Technik zum Multiplexen digitaler Datenströme
 Pyruvatdehydrogenase E1, ein Teil eines großen Multienzymkomplexes, beteiligt an biochemischen Stoffwechselreaktionen
 Pound-Drever-Hall-Verfahren, eine Technik zur Stabilisierung von Laserfrequenzen

PdH steht für:
 Partei der Humanisten, eine deutsche Kleinpartei

PDH

redpajama_set_name: RedPajamaWikipedia


I would like to give you a feedback about my order for the Bach flower mix 90 for our little Claire. For about 1-2 months she had a lot of difficulties getting to sleep. A lot of jolts which were preventing her to maintain a deep and restorative sleep were made worse by a series of inconsolable crying fits.
These were especially intense and going on during the day and night. Since the first intake of Bach flower remedies, our daughter found serenity and she's able to have a deep and calm sleep. We had the happiness of discovering, day by day, a sociable, smiling, and rested baby who is now able to fall asleep alone and everywhere, even with other kids.
Life is beautiful! We will keep giving the Bach flower remedy until the end of the three months as you advised us to do, to ensure the stabilisation of her state. We thank you for your help and the fast order processing done from your website in Switzerland, and we only can recommend to other parents the use of Bach flowers remedies proposed for this kind of situation.
Best regards and good end of week to you.


Just because you can navigate a website, it doesn't mean you can build one. You need some coding skills to tinker with various design features and HTML. If you want your site to look great, learn about web design. Make sure to check out the tips included here if you need help.
You want to make your site content compelling and intriguing. The appearance of the site design is crucial, but content keeps visitors coming back. If you have insightful, useful content that people want to read about, you are much more likely to have repeat visitors.
Be vigilant in policing your website for content that has become obsolete or outdated. When you are offering specials that are past their expiration, you will watch your visitor counts dwindle. Users wants to browse sites that they know are updated frequented with relevant fresh content, but if the site has outdated information then they know it is not well-maintained. Put removing outdated content on your to-do list, to do every 2-4 weeks.
When deciding on a background color for your website, white is a very effective and good choice. Few visitors take issue with a white background, which looks professional or, at worst, neutral. Text content is also more clear on a white background. Complicated background designs, however, can be distracting, or make your site appear amateur. When choosing a background for your website, a simple background is better.
Take advantage of as many shortcuts as you can. In web design, there are lots of shortcuts that will decrease the amount of time it takes to do things. You can even use HTML codes to help make changes quicker so that your pages don't need to be uploaded again.
Be mindful of what particular file type you use for graphics, since that impacts file sizes and load times for users. GIF's are much easier to utilize on your site. Web graphics look better when they are made from PNG and BMP files, but they take up much more disk space. Convert your images to file types that are smaller and easier to manage if you want a guaranteed great user experience.
Make sure that your page loads in less than 10 seconds. Sites that are well-made and efficient will open in a browser in moments. If it takes too long, a visitor is simply going to become frustrated and go elsewhere.
All load times on the website you design should be as fast as possible. If visitors must wait a long time for something to load, they will want to leave the site. Do your best to reduce graphics, scripts, and Flash.
Encourage visitors to come back to a website repeatedly, by offering them a newsletter. When you allow people to receive notifications about special events, discounts or updates, they are more likely to visit your site multiple times. Add a signup bar on your site and use it to track the number of visitors who do so. Make sure you only send out the letter to those that have explicitly requested to view it.
Reserve a domain as soon as you can. Use your creativity, and pick out a name that you want. Reserve it now before someone else does. You'd be surprised at just how many people have the same ideas. It's like everyone is connected or something similar.
Your front page should be simple. People who are shopping around decide quickly whether to use a site or continue searching by viewing the front page. Be descriptive about what your business does, and your unique offerings, but keep everything else to a minimum to avoid distraction.
For creating your own website, consider using Adobe Dreamweaver. Even if you are a beginner, you can easily use this program. It also includes great templates to make your site look amazing.
Make sure your webpages are designed in a way that is easy for people to navigate through. The placement of your navigation links on a website plays an important part in determining how long a visitor will remain on your site. Navigating your site must be easy, consistent, and neat for visitors to have a good experience.
Make an effort to create a high quality "About Us" webpage. There are a lot of websites that do not utilize compelling or original content for their "About Us" webpages. This is an opportunity to make your website stand out. Let people see a bit of your personal history. Try including items like, how you entered website creation, who inspires you, and what your wishes for your business are.
Task based usability tests are a great way to determine how effective the design of your website is. These tasks help to find the information that is in your site. A well-designed site will make it easy for the user to complete the task at hand. If the user has trouble, it can provide you with valuable insight into areas where you can improve.
You should have some place exclusive within which to work. Remove distractions and set up your work station in a way that allows you to jump into work quickly, and accomplish tasks efficiently. Have your office tools and everything else located where you have easy access, and make sure you have an optimized space for your website design needs.
Implement site searching abilities into your home and sub pages. This is absolutely essential for maximizing the site's usability, and creates easy access to whatever information the viewer is seeking. It is pretty simple to put a site search in place. The time spent will be well worth it.
Make sure your site's design isn't too similar to other sites in your niche. Check around and visit similar websites. You want to be separate from those in the pack, so you don't want your website too similar to someone else's. You will only be seen as a copycat of the one that was up previously.
Remember, designing your site needs not to cost a lot of money. For each costly, popular program or tool available for web design, there are equally useful programs or tools that are much cheaper. For example, open source software can accomplish many of the same things you might use paid software for, and it is free. Using this open source software may save you a lot of money.
Building a website does not dictate you need to purchase books. Although you may think it is a wise idea, it isn't. You can find books available that are filled with useful information. However, much of this is available at no cost. Paying for information is no better than information than can be obtained for free.
Take literacy levels into account when you write site content. A variety of visitors will visit your site and you want everyone to have access to the information shared. To grow your audience, create easy-to-understand content.
Website content should be available to all people. It is always advisable to look for feedback from individuals who may be located in another part of the world. Content display may vary per country, so be sure to have it checked.
If you keep learning as you are designing a website, that will be a benefit to you. Once you master one facet of web design, move on to the next. While this may lengthen the process of building your first site or two, you will soon have the knowledge you need to crank out websites much faster.
Flash is not something you want to use on your site. While it is indeed flashy, it can cause your site to slow down, crash your reader's browser, or even not show up at all. There are many tablets and phones that are not Flash enabled, and if people are using them to view your site, it can ruin their experience, and make them not come back again.
When you design any website, make certain that everyone can access it. So be sure to have a few people from many different locations access it and test it for you. Because some content may fail to display properly in foreign countries, it is best to check on this.
Use user-friendly tools. Ensure your underlined text is clickable and make sure they will change color once they are clicked. If you do these things, your visitors will stay on your site longer as well as return for more info at a later date.
If you are going to feature advertisers on your website, keep them small and not huge or distracting. People should feel comfortable visiting your site instead of bombarded by ads. Therefore, add content that will not look like clutter on your website, and your visitors will be so impressed by how professional it looks that they will return.
Link your logo to your home web page. Over the years most people have come to associate a logo with a link, that when clicked, will take them to the homepage of a website. It can prove frustrating to your users if they click on a static logo, because then they have to search to find a 'home' link. Your site's navigation and ease of use will be much improved by making your logo into a link.
If you have a search text box on your website, allow at least 30 characters to be visible. Be sure that the search button is labeled "search", rather than "go or "submit." Following these guidelines ensures that site visitors can quickly find and use this feature to locate information.
Avoid using Flash on your website. While Flash can look very high-tech and exciting, on some devices it will cause everything to slow down. Don't forget that many phones and tablet PCs don't have Fl;ash enabled, which can ruin the user experience for some and dissuade them from returning to your site.
Be sure the title of your site reflects what it is about. You can check out the implications of this by searching for an "untitled document" on search engines. As you will see, page titling is a step that all too may web designers neglect! The importance of naming your site cannot be overstated. This will play a large factor in how you rank on search engines.
Because technology is advancing so rapidly, it is critical that you stay ahead of the game when it comes to web design. Websites are a great way for companies to get their information out there. Many business owners have found that having a website can help them earn more money than ever before.
You must put some effort into your website' logo, because the logo carries out an important function. Your logo has to make you stand out from the crowd, but also convey what you are about. If you manage to hit a creative wall, it happens, seek assistance form a logo design specialist in order to create the logo you need to represent your success.
Carve out some time to build your website each day. You need to dedicate extended periods of time so that you can actually get some work done rather than just working a little bit here and there. Working hard on web design will help you build a nice site.
After reading the information regarding web page design, clearly it is not too difficult to design a site. You should have the right information to help guide you. Create the perfect site by using the above tips.
A cc (country code) top level domain is useful to buy when you want traffic from a certain area. This can guarantee the domain name is just for a particular region in the country. CCTLDs are great for directing visitors from different regions to your website, especially when they might not find it otherwise.


text:
SpaceX Is About to Win Its First-Ever U.S. Military Mission
Kaila Hale-Stern
It looks like SpaceX is set to land its first US military contract. Elon Musk's company sued for the right to be considered, and now they're the last ones standing after United Launch Alliance's Russian-made engines have become an issue.
The contract at hand is a next-gen GPS satellite for the U.S. Air Force. United Launch Alliance, a joint venture of Lockheed Martin and Boeing, has dominated the military satellite scene since 2006, but this year is different; new laws passed by Congress in response to the ongoing tensions between the United States and Russia will ban the use of the Russian-made RD-180 engine, which ULA's rockets rely on.
Ars Technica reports:
ULA's chief executive, Tory Bruno, told the Washington Post that his company was "unable to submit a compliant proposal" because of contract requirements and the limitations Congress has imposed on the RD-180 engine.
The company's decision appears to bring some closure to the public and legal battle SpaceX launched in April, 2014, to buttress its efforts to get into the national defense satellite launch market. At the time, SpaceX CEO Elon Musk accused the US government of entering into improper and uncompeted launch agreements with private contractors and said that SpaceX's rockets could do the same job at a far lower cost. "We're just protesting and saying these launches should be competed," said Musk at a press conference. "And if we compete and lose, that's fine, but why were they not even competed?"
With ULA bowing out, SpaceX is poised to step up on the military space stage. The contract will be a big win for Musk and his rising firm—military contracts are hugely lucrative, not to mention that this marks the successful end to a long fight to be considered a contender.
[Ars Technica]
Top image via SpaceX.

redpajama_set_name: RedPajamaCommonCrawl


Victim: Male. Mid-thirties. 5'7". Cause of death: Hanging. Initial impression - murder. ID: Mickey Doyle. Suspected terrorist and member of the Mayday Five. The officers at the crime scene know exactly who the victim is. Doyle was one of five suspected bombers who caused the deaths of sixteen people. The remaining four are also missing and when a second body is found, decapitated, it's clear they are being killed by the same methods their victims suffered. Forensic psychologist Paula Maguire is assigned the case but she is up against the clock - both personally and professionally. With moral boundaries blurred between victim and perpetrator, will be Paula be able to find those responsible? After all, even killers deserve justice, don't they?
The Silent Dead, by Claire MCGowan, is the third book to feature forensic psychologist Paula Maguire who is working for the police in the missing persons unit. Paula, now pregnant and uncertain who the father is, is working on a case of 5 people who appear to have been abducted. All 5 were cleared of a bomb attack (post the Good Friday agreement) in the small town which caused many deaths & injuries and left many families affected. The story here deals with both the consequences of the modern bombing and the legacies of the past. In a sense this book focuses of the remains of the "Troubles" in the present day. With echoes of the old troubles and more current splinter group issues there seems to be a fairly accurate portrayal of the Northern Ireland of the present.
I missed the 2nd book in this series but I like Paula and I do like the writing so I'll put that right soon. The book feels atmospheric and thoughtfully written. However it is also dark – bleak & violent at times – and gritty. The use of the counterpoint of then and now is effective. There is a running theme in this book and the first (& I assume the second) in that, while Paula is working in the missing persons unit. She and her family have also been affected by some issues and she herself is on a quest for knowledge. I find this very well done indeed. I guess it goes without saying that she is a bit of a maverick character doing as she sees fit despite the views of those in authority however Claire McGowan writes her well and I certainly find Paula an appealing character. I'd suggest that the writing is getting better in these books. The characters are becoming fuller and more rounded and, for me, they make for good gritty crime reading in a believable setting. I'd like to read the next book and will try and read book 2 as well. I guess they could be read standalone however it would probably be better to read them in order. I'd probably say 4.5/5 all in all.


iPad mini(R) 4 16GB WiFi Space Grey (GameStop Premium Refurbished) by GameStop Inc.
iPad mini(R) 4 16GB WiFi Space Grey (GameStop Premium Refurbished) is rated 4.7 out of 5 by 3.
This iPad mini 4 16GB WiFi Space Grey (GameStop Premium Refurbished) is more than just a tablet. Watch movies, listen to your favorite songs, play games and more! Purchase includes a new USB sync cable and new AC adapter.


The $1million ATC Golden Rose is the first Group I of the Australian racing season and Coolmore sires filled the quinella with Manawanui (3g Oratorio x Lady Remlap, by Filante) defying the late challenge of boom colt Smart Missile (3c Fastnet Rock x Comical Smile, by Comic Strip). Trained at Warwick Farm by Ron Leemon, Manawanui has been in spectacular form in recent starts albeit in lesser grade and with a gem of a ride from Glynn Schofield was able to blow the big guns out of the water in this lucrative 1400 metre Rosehill feature. When dual Group I winner Helmet was slowly away and nobody really wanted to lead, Schofield seized the initiative and allowed Manawanui to stride forward and that was where the race was won. With interference back in the field hampering both Smart Missile and Helmet, Manawanui stole a march on his rivals to hold a short neck margin on the line over Smart Missile, who charged home out wide late for an unlucky second. "Everything went wrong at the start and when he went to the front, I said, 'oh no' but it's full credit to Glyn, he rode a tremendous race," said Ron Leemon, who has steadfastly refused all offers for his stable star including one for $1million on the eve of the race. Manawanui has now won four of five starts earning $788,380, so his decision looks well founded. ''I was always going to ride him positively and I thought if I was in the position Foxwedge was in the other day, that my horse would have won that race,'' said winning rider Glynn Schofield. ''My bloke kicked like I expected, when he [Smart Missile] came at him, so really the race was over when I found the front.'' A $45,000 purchase for his trainer from the Goodwood Farm draft at the 2010 Inglis Classic Yearling Sale, Manawanui is the second foal and second winner for seven-time metropolitan winner Lady Remlap, whose previous foal is the good metropolitan winner La Remlap. A daughter of Epsom Handicap winning sire Filante, Lady Remlap has been a regular visitor to Coolmore and has a two year-old filly by Choisir, the sire of La Remlap, and a yearling colt by Oratorio. Manawanui is the first stakes-winner in his family in five generations, so Oratorio has certainly done a good job of upgrading this old Australian pedigree, which traces back to champion filly Wenona Girl, who is his sixth dam. A triple Group I winning son of Danehill, Oratorio returns to Australia this spring at a bargain fee of $11,000 and Manawanui is his 13th stakes-winner worldwide and third Group I winner joining Banchee (New Zealand) and Beethoven (UK). Oratorio went close to making it a stakes double with his high class four year-old Torio's Quest (4g Oratorio x Quest for Pleasure, by Quest for Fame) rattling home for a close second in the Group II ATC Sebring Stakes at his first run back from a break.


text:
Criminal charges against PG&E possible if utility is found responsible for recent wildfires, prosecutors say
Cars sit among one of the many commercial businesses destroyed in Paradise, Calif., by the Camp fire.
(Peter DaSilva / EPA/Shutterstock)
By Alene TchekmedyianStaff Writer
Pacific Gas & Electric Co. could be charged with murder or involuntary manslaughter if authorities determine that recent deadly California wildfires ignited as a result of the "reckless" operation or maintenance of power lines, state prosecutors said.
In a court filing Friday, California Atty. Gen. Xavier Becerra's office detailed potential criminal charges the utility could face if found responsible for wildfires in the state that have killed dozens of people and destroyed thousands of homes; possible charges include failing to keep power lines clear of trees or vegetation, recklessly starting a forest fire and murder.
A spokesperson for the utility released a statement Sunday in response to the filing but declined to comment further.
"PG&E's most important responsibility is public and workforce safety. Our focus continues to be on assessing our infrastructure to further enhance safety and helping our customers continue to recover and rebuild," the statement said. "Throughout our service area, we are committed to doing everything we can to help further reduce the risk of wildfire."
The attorney general's filing came at the request of a federal judge overseeing the utility's probation in a criminal case stemming from the deadly San Bruno gas pipeline blast in 2010, which killed eight people. PG&E was sentenced to five years' probation and ordered to pay a $3-million fine in that case.
U.S. District Judge William Alsup asked Becerra's office to identify each potential criminal offense the utility could face in connection with the blazes.
He also asked the utility, federal prosecutors and an independent monitor reviewing the company's safety performance as part of the San Bruno case to weigh in on PG&E's actions since the sentencing, setting a Monday deadline. As of Sunday night, only Becerra's office had filed a response.
Laurie Levenson, professor at Loyola Law School, said the severity of charges the utility faces depends on whether it was "on notice of the risk that their equipment may start the fires and lead to deaths."
"Callous disregard of that risk could be murder," she said. "The more likely charge would probably be involuntary manslaughter, which requires criminal negligence…. The real key here is how much risk they realized they were taking and why they took it."
Though companies can't be jailed if held criminally liable, they can be fined and debarred from contracts. When that happens, it is like a corporate death penalty, Levenson said.
State prosecutors said the utility's criminal liability would depend on its degree of recklessness.
" 'Recklessness' could include the mental state of multiple offenses. These mental states resemble a sliding scale. As the mental state becomes more culpable, the applicable offense becomes more serious," the filing said.
It continued: "At criminal negligence, PG&E could have committed involuntary manslaughter, starting a fire without permission, and failing to keep its lines and poles clear of vegetation. At recklessness … PG&E could have committed the felony of unlawfully starting a fire. And at malice, PG&E could have committed murder."
Butte County Dist. Atty. Mike Ramsey told the Sacramento Bee that he hasn't decided whether to file criminal charges against PG&E over the Camp fire, even if the California Department of Forestry and Fire Protection determines the company is at fault.
"It's a little premature," he said.
In the filing, state prosecutors don't offer a position on PG&E's criminal liability or whether the utility caused the fires. But PG&E is facing intense criticism and at least 20 lawsuits that allege the utility allowed its equipment to spark the deadliest wildfire on record in California history.
One lawsuit points to a small metal hook on an electrical transmission tower as the cause in the Camp fire, which killed 86 people and displaced up to 50,000 in Butte County. When it failed, the suit alleges, an uninsulated wire touched the tower, caused sparks and ignited the blaze.
Another blasts a corporate culture that allegedly places "reputation above public safety" along with advertising that "promotes a false and misleading picture" of Northern California's electricity supply.
Other suits blame the general decay of the region's electrical infrastructure and the failure to trim trees and brush that grew too close to poles and power lines.
alene.tchekmedyian@latimes.com
Alene Tchekmedyian
Alene Tchekmedyian is an investigative reporter at the Los Angeles Times. She previously covered the Los Angeles County Sheriff's Department, focusing on accountability stories and writing about failures by officials to comply with transparency laws. Before joining The Times in 2016, she reported on crime and policing for the Glendale News-Press and Burbank Leader.


text:
Biography - FRITZ CHARLES NOEL
Fritz Charles Noel, editor and part owner of the Leland Tunes, is a native of Stockholm, where his father, Fredrik Adolf Noel, was a manufacturer. The younger Noel was born May 11, 1867, and educated at the Hernosand Seminary, at the elementary school at Ladugardslandet in Stockholm and at the City of London College.
In 1888 he emigrated, going to Montreal, Canada, where he remained for three years, and began his journalistic career with the Montreal Herald. Ten years were next spent in Chicago in the service of the Chicago Tribune. Five years ago he removed to Leland and became interested in the local newspaper, the Times, the editorship of which he has since assumed. Besides his interest in this paper, he is a member of the Leland Publishing Company, publishers of art calendars for the trade.
Mr. Noel has not held political office, but is active in local political affairs, and that his opinion carries weight may be inferred from the fact that he has been called in as advising member of the County Central Committee.
He is a Lutheran and a member of the Bethany Church, together with his family, consisting of his wife, Martha Audrine, nee Klove, whom he married Sept. 18, 1895, and two children, Gladys Josephine, born 1898, and Fredrik Andrew, born 1906.
Extracted 17 Sep 2016 by Norma Hass from the History of the Swedes of Illinois, published in 1908 by Engberg-Holmberg Publishing Company, Part 2, page 209-210.


text:
Q: $(2, 1 + \sqrt[]{−17})$ is a prime ideal in $\Bbb Z[\sqrt[]{-17}]$ How can we see that $(2, 1 + \sqrt[]{−17})$ is prime ideal in $\Bbb Z[\sqrt[]{−17}]$?
We have \begin{align*}
\frac{\mathbb{Z}\left[\sqrt{-17}\right]}{\left(2, 1 + \sqrt{-17}\right)} &\cong \frac{\mathbb{Z}[x]/(x^2 + 17)}{(2, 1 + x, x^2 + 17)/(x^2 + 17)} \cong \frac{\mathbb{Z}[x]}{(2, 1+x, x^2 + 17)}.
\end{align*}
I don't see why this is an integral domain?
Thank you for your help.

A: $$\frac{\Bbb Z[\sqrt{-17}]}{(2,1+\sqrt{-17})}\simeq\frac{\frac{\Bbb Z[\sqrt{-17}]}{(2)}}{\frac{(2,1+\sqrt{-17})}{(2)}}\simeq\frac{\Bbb Z_2[i]}{(1+i)}\simeq\Bbb Z_2.$$
Here I have used, Third Isomorphism theorem and reduction modulo $2$.

A: A general element of $\Bbb{Z}[\sqrt{-17}]$ is $a + b\sqrt{-17}$ for $a,b \in \Bbb{Z}$.  When you mod out by $(1+\sqrt{-17}) \subset (2,1+\sqrt{-17})$, you impose the relation $\sqrt{-17} = -1$, so
$$  a + b\sqrt{-17} \cong a-b \pmod{(1+\sqrt{-17})}  \text{.}  $$
This makes $\frac{\Bbb{Z}[-17]}{(1 + \sqrt{-17})}$ (a subring of) $\Bbb{Z}$.  (There's a reasonable chance that $(1 + \sqrt{-17})$ contains an integer, perhaps something near $18$, so we may actually have a much smaller ring than $\Bbb{Z}$.)  Then when we quotient by $(2)$, we get either $\{0\}$ (a.k.a., the one ring) or $\Bbb{Z}/2\Bbb{Z}$.  The former happens when we actually have $\Bbb{Z}$ mod an odd number (because having both $\text{"some odd integer"} \cong 0$ and $2 \cong 0$ sends all of $\Bbb{Z}$ to zero) and the latter when we actually have $\Bbb{Z}$ or $\Bbb{Z}$ mod an even number after the first quotient.
If you can show $(2, 1+\sqrt{-17})$ is a proper ideal, you do not get the one ring.  So you get a field, showing that $(2, 1+\sqrt{-17})$ is a maximal ideal.  Since maximal ideals are prime ideals, $(2, 1+\sqrt{-17})$ is a prime ideal of $\Bbb{Z}$.
Can show $(2, 1+\sqrt{-17})$ is a proper ideal?

A: Modding out by $x+1$ is the same as setting $x = -1$. Or more precisely, using the Third Isomorphism Theorem
\begin{align*}
\frac{\mathbb{Z}[x]}{(2, 1+x, x^2 + 17)} & \cong \frac{\mathbb{Z}[x]/(1+x)}{(2, 1+x, x^2 + 17)/(1+x)} \cong \frac{\mathbb{Z}[-1]}{(2, (-1)^2 + 17)} = \frac{\mathbb{Z}}{(2, 18)} = \frac{\mathbb{Z}}{(2)} \, .
\end{align*}


One of our most popular Rex Bakery products, Benedict Bars apparently originate from a South African cake. Our Rex version's shortbread biscuit base is hand crumbled for a better rise, with seedless raspberry jam & topped with a heated milk, sugar, butter, vanilla & flaked almond mix which toasts during baking, and all combines irresistibly for nutty, jammy, wholesome goodness.
No words can describe this, you just have to try it!
Heavenly, can't be beaten in my opinion and just has to be tried to be believed!
This is absolutely delicious, a slice of heaven and always has been a firm favourite of mine.


text:
Reports: Nokia says no Android phone in the works
By Daniel Ionescu, Macworld	 Jul 6, 2009 2:47 am PDT
The following article is reprinted from the Today@PC World blog at PCWorld.com.
Nokia has denied today reports that the Finnish company is working on a mobile phone running on Google's Android mobile operating system.
A report Monday morning from the Guardian, a British daily newspaper, quoted "industry insiders" saying that Nokia would unveil in September a mobile phone running on the Google Android operating system (OS).
But Nokia has denied Guardian's speculation, telling Reuters that there is "absolutely no truth to this whatsoever." The denial continued quoting a company spokesman saying: "Everyone knows that Symbian is our preferred platform for advanced mobile devices."
Indeed, Symbian is Nokia's OS of choice for smartphones, or multimedia computers, as the company likes to call its devices. But Nokia's popularity has been dwindling in recent years, and HSBC analysts quoted by the Guardian reckon that the Finnish company has lost smartphone market share, going down to 31 percent at the end of 2008 (from 47 percent in 2007).
However, Nokia has invested heavily in Symbian in recent years. The biggest move we saw was the company buying out the partners in the Symbian OS joint venture last year. With this, Nokia planned to offer the platform free of charge to other smartphone manufacturers.
And if this sounds similar, then it's because Google is using the same approach with its Android mobile OS. The search giant is offering Android for free to any manufactures that want to release smartphones using its OS. So far, this model proved to be increasingly popular, with plenty of Android cell phones coming out this year (T-Mobile myTouch 3G, Samsung Galaxy, HTC Hero—to name a few).
But while Google's Android OS was built from the ground up with touchscreen use in mind, Nokia's latest devices featuring the Symbian OS (the flagship N97 and the 5800 model) have been criticized for the lack of finger-friendly navigation. Symbian, originally designed for devices with alphanumeric keyboards, and then revamped for touchscreen phones, still requires the use of a stylus in many cases.
Perhaps Guardian's "industry insiders" are not entirely wrong though. Late last month, Nokia partnered with Intel to create a new stable of mobile products. While Nokia has the Linux-based Maemo OS, which powers the company's Internet tablets, Google Android could be the company's OS of choice for this new breed of devices.


text:
Arlington City Council approves master agreement for PPP with Rangers to construct new US$1bn ballpark
You are here: Home / Latest news / Arlington City Council approves master agreement for PPP with Rangers to construct new US$1bn ballpark
Tags: Globe Life Park, new ballpark, new stadium proposals, Texas Rangers
Arlington City Council has unanimously approved a master agreement for a public-private partnership (PPP) with the Texas Rangers for the possible construction of a US$1bn retractable-roof stadium and supporting development in the Entertainment District at its May 24 meeting.
This agreement allows the City of Arlington to begin the steps required to hold an election this November that asks voters to extend existing funding mechanisms that would provide the public financing for construction of a new ballpark. The Texas Rangers' 30-year lease on the City-owned Globe Life Park in Arlington is set to end in 2024. With the new master agreement, the Rangers' partnership with Arlington would extend until January 1, 2054.
The design for the new ballpark will include a retractable roof for climate control and shelter for fans during the hot summer months, as well as a premiere baseball experience. The new park is expected to have similar design concepts featured in the construction of Globe Life Park.
While designs and costs have not been finalised, the cost of the new ballpark and related infrastructure is estimated at US$1bn. The master agreement for this PPP calls for a 50-50 split between the Texas Rangers and the City of Arlington.
Arlington Mayor W. Jeff Williams, said:
The Texas Rangers are a part of our fabric, a part of our DNA. The City Council and I have heard the message loud and clear: our citizens want the Texas Rangers to stay in Arlington. No doubt any city would want the Rangers. We want to be proactive and continue this 45-year relationship, which has been so successful for us, for the team and the North Texas region. This is where they belong and this deal keeps them here for generations to come.
Texas Rangers Co-Chairman, Ray Davis, issued the following statement after the Council unanimously approved the master agreement:
Tonight's action by the City Council is an important step in the process in this public-private partnership to build a new ballpark for Arlington and all our fans. I want to thank and commend Mayor Jeff Williams and the Arlington city leaders for their vision in reaching out to us and helping put together and approving this historic agreement. The Rangers are excited about the possibility of calling Arlington home for many years to come, and we are committed to building a world-class facility which will provide the best possible experience for our fans. We look forward to working with Mayor Williams, the City leaders, and the citizens of Arlington over the next several months to insure that this dream becomes a reality.
The master agreement caps Arlington's proposed bond-funded contribution for the ballpark and outlines the City's requirements and expectations for the project to move forward.
The City will now ask the Texas Office of the Comptroller to review the fiscal impact to the State before calling a November 8 election on the proposed methods of financing.
The Arlington Convention and Visitors Bureau commissioned an independent consulting group to provide an analysis of the economic impact of the Texas Rangers franchise on Arlington. HR&A Advisors found that the annual economic impact of the Rangers in a new ballpark would be US$74m for Arlington and US$133.5m for Tarrant County. According to HR&A's analysis, the net present value of the Rangers continued presence between 2016-2054 with a new ballpark would be US$2.43bn for Arlington and US$4.35bn for Tarrant County.
The proposed new stadium will be built on what is currently a team parking lot, south of the existing ballpark, south of Randol Mill Road. The City and the Rangers are discussing the future of the 23-year-old Globe Life Park in Arlington, with options including office development, park space, parking and a possible extension to the Texas Live! project.
Arlington voters have previously supported stadium projects; with voters having approved a half-cent sales tax to help construct the Texas Rangers' current home back in 1991. That sales tax went away after the City's US$135m debt on the ballpark was paid off in 2001, a decade earlier than scheduled. Then again in 2004, voters approved a half-cent sales tax, a 5 percent car rental tax and a 2 percent hotel occupancy tax (venue taxes) to help publically finance the Dallas Cowboys' US$1.2bn AT&T Stadium. The City's stadium debt is scheduled to be paid off in 2028 but is projected to be paid off at a faster pace.
The City of Arlington will now set about putting in place the necessary steps to hold an election on November 8, 2016, and establish the possible funding mechanisms to help finance the new ballpark's construction, while also paying off the AT&T Stadium debt.
Image: Texas Rangers' current 'home', Globe Life Park. Courtesy: Texas Rangers.
Trivandi is three: Success goes on for London 2012 senior venues team
FabriTec Structures and Guard-All Building Solutions partner with PFEIFER's business unit, Cable Structures