text:
\section{Introduction}\label{sec:intro}

Consider a setting where we have iid observations from a mixture
model.  More specifically, let $G_0$ be a probability distribution on
$\mathcal{T} \subseteq \mathbb{R}^d$ and let $\{
F_0(\cdot\mid\theta)\}_{\theta \in \mathcal{T}}$ be a family of probability
distributions on $\mathbb{R}^n$ indexed by
the parameter $\theta \in \mathcal{T}$.  Throughout the paper, we assume
that $\mathcal{T}$ is closed and convex.  Assume that $X_1,...,X_p
\in \mathbb{R}^n$ are observed iid random variables and that $\Theta_1,...,\Theta_p \in
\mathbb{R}^d$ are corresponding iid latent variables, which satisfy
\begin{equation}\label{MM}
X_j\mid\Theta_j \sim F_0(\cdot\mid\Theta_j) \mbox{ and } \Theta_j \sim G_0.
\end{equation}
In \eqref{MM}, it may be the case that $F_0(\cdot\mid\theta)$ and
$G_0$ are both known (pre-specified) distributions; more frequently,
this is not the case.  In this paper, we will study problems where the mixing distribution $G_0$ is
unknown, but will assume
$F_0(\cdot\mid\theta)$ is known throughout.  Problems like this arise in
applications throughout statistics, and various solutions have been proposed.
The distribution $G_0$ can be modeled parametrically, which leads to
hierarchical modeling and parametric empirical Bayes methods \cite[e.g.][]{efron2010large}.  Another
approach is to model $G_0$ as a discrete distribution supported on
finitely- or infinitely-many points; this leads to the study of
finite mixture models or nonparametric Bayes, respectively \citep{mclachlan2004finite,ferguson1973bayesian}.   This
paper focuses on another method for estimating $G_0$: Nonparametric maximum
likelihood.

Nonparametric maximum likelihood (NPML) methods for mixture
models --- and closely related empirical Bayes methods --- have been studied in statistics since the 1950s
\citep{robbins1950generalization, kiefer1956consistency,robbins1956empirical}.  They make
virtually no assumptions on the mixing distribution $G_0$ and provide
an elegant approach to problems like \eqref{MM}.   The general strategy is to first find the nonparametric maximum likelihood estimator (NPMLE) for $G_0$, denoted by $\hat{G}$, then perform inference via empirical Bayes \citep{robbins1956empirical,efron2010large}; that is, inference in \eqref{MM} is conducted via the posterior distribution $\Theta_j\mid X_j$, under the assumption $G_0 = \hat{G}$.
Research into NPMLEs for mixture models has included
work on algorithms for computing NPMLEs and theoretical work on their
statistical properties \citep[e.g.][]{laird1978nonparametric,bohning1992computer,lindsay1995mixture,ghosal2001entropies,jiang2009general}.  However,
implementing and analyzing NPMLEs for mixture models has
historically been considered very challenging  \citep[e.g. p. 571
of][]{dasgupta2008asymptotic,donoho2013achieving}.  In this paper, we
study a computationally convenient approach involving approximate NPMLEs, which sidesteps many
of these difficulties and is shown to be effective in a variety of applications.

Our approach is largely motivated by recent work initiated by \cite{koenker2014convex}\footnote{Koenker \& Mizera's work was itself partially inspired by relatively recent theoretical work on NPMLEs by \cite{jiang2009general}.} and further pursued by others, including \cite{gu2015problem, gu2017empirical, gu2017unobserved} and \cite{dicker2016nonparametric}.  Koenker \& Mizera studied convex approximations to
NPMLEs for mixture
models in relatively large-scale problems, with up to 100,000s of observations.  In
\cite{koenker2014convex}, they showed that for the Gaussian location model, where $X_j =
\Theta_j + Z_j \in \mathbb{R}$ and $\Theta_j \sim G_0$, $Z_j \sim N(0,1)$ are
independent, a good approximation to the NPMLE for $G_0$ can be
accurately and
rapidly computed using generic interior point methods.

\cite{koenker2014convex}'s focus on convexity
and scalability is one of the key concepts for this paper.  Here, we show how a simple convex approximation to the NPMLE can
be used effectively in a broad range of problems with nonparametric mixture models; including problems
involving (i) multivariate mixing distributions, (ii) discrete data,
(iii) high-dimensional classification, and (iv) state-space models. Backed by new theoretical and empirical results, we provide concrete
guidance for efficiently and reliably computing approximate multivariate
NPMLEs.  Our main theoretical result (Proposition \ref{prop:conv}) suggests a simple procedure for finding the support set of the estimated mixing distribution.  Many of our empirical results highlight the benefits of using multivariate mixing distributions with correlated components (Sections \ref{sec:numMean}, \ref{sec:bball}, and \ref{sec:class}), as opposed to univariate mixing distributions, which have been the primary focus of previous research in this area (notable exceptions include theoretical work on the Gaussian location-scale model in \cite{ghosal2001entropies} and applications in \cite{gu2017empirical,gu2017unobserved} involving estimation problems with Gaussian models).  In Sections \ref{sec:bball}--\ref{sec:cgm}, we illustrate the performance of the methods described here in real-data
applications involving baseball, cancer microarray data, and online
blood-glucose monitoring for diabetes patients.  In comparison with other recent work on NPMLEs for mixture models, this paper distinguishes itself from \cite{gu2015problem} in that it focuses on more practical aspects of fitting general multivariate NPMLEs.  Additionally, in this paper we consider a substantially broader swath of applications than \cite{gu2017empirical,gu2017unobserved} and \cite{dicker2016nonparametric}, where the focus is estimation in Gaussian models and classification with a univariate NPMLE, respectively, and show that the same fundamental ideas may be effectively applied in all of these settings.

\section{NPMLEs for mixture models via convex optimization}
\label{sec:NPMLE}

\subsection{NPMLEs}
\label{sec:NPMLEsub}

Let $\mathcal{G}_{\mathcal{T}}$ denote the class of all probability distributions on
$\mathcal{T} \subseteq \mathbb{R}^d$ and suppose that $f_0(\cdot\mid\theta)$ is the
probability density corresponding to $F_0(\cdot\mid\theta)$ (with respect to
some given base measure).  For $G \in \mathcal{G}_{\mathcal{T}}$, the (negative) log-likelihood given the
data $X_1,...,X_p$ is
\[
\ell(G) =  -\frac{1}{p}\sum_{j = 1}^p \log \left\{\int_{\mathcal{T}}
f_0(X_j\mid\theta) \ dG(\theta)\right\}.
\]
The Kiefer-Wolfowitz NPMLE for $G_0$ \citep{kiefer1956consistency},
denoted $\hat{G}$, solves the optimization problem
\begin{equation}\label{KW}
\min_{G \in \mathcal{G}_{\mathcal{T}}} \ell(G);
\end{equation}
in other words, $\ell(\hat{G}) = \min_{G \in \mathcal{G}_{\mathcal{T}}}
\ell(G)$.

Solving
\eqref{KW} and studying properties of $\hat{G}$ forms the basis for
basically all of the existing research into NPMLEs for mixture models
(including this paper).  Two important observations have had significant but somewhat
countervailing effects on this research:
\begin{itemize}
	\item[(i)] The optimization problem
	\eqref{KW} is convex;
	\item[(ii)] If $f_0(X_j|\theta)$ and $\mathcal{T}$ satisfy certain
	(relatively weak) regularity conditions, then $\hat{G}$ exists and may be chosen
	so that it is a discrete measure supported on at most $p$
	points.
\end{itemize}
The first observation above is obvious; the second
summarizes Theorems 18--21 of \cite{lindsay1995mixture}. Among the
more significant regularity conditions mentioned in (ii)
is that the set $\{f_0(X_j|\theta)\}_{\theta \in \mathcal{T}}$ should be bounded for
each $j = 1,...,p$.

Observation (i) leads to KKT-like conditions that characterize
$\hat{G}$ in terms of the gradient of $\ell$ and can be used to develop
algorithms for solving
\eqref{KW} \citep[e.g.][]{lindsay1995mixture}.  While this approach
is somewhat appealing, \eqref{KW} is typically an
infinite-dimensional optimization problem (whenever $\mathcal{T}$ is
infinite).  Hence, there are infinitely
many KKT conditions to check, which is generally impossible in
practice.

On the other hand, observation (ii) reduces \eqref{KW} to a
finite-dimensional optimization problem.  Indeed, (ii)
implies that $\hat{G}$ can be found by restricting attention in
\eqref{KW} to $G \in \mathcal{G}_p$, where $\mathcal{G}_p$ is the set of discrete probability measures
supported on at most
$p$ points in $\mathcal{T}$.  Thus, finding $\hat{G}$ is reduced to
fitting a finite mixture model with at most $p$ components.  This is
usually done with the EM-algorithm \citep{laird1978nonparametric}, where
in practice one may restrict to $G \in \mathcal{G}_q$ for some $q < p$. More recently, \citet{wang2015nonparametric} proposed to estimate $G$ in a multivariate setting using a gradient based method.
However, while (ii) reduces \eqref{KW} to a
finite-dimensional problem, we have lost convexity:
\begin{equation}\label{ncvx}
\min_{G \in \mathcal{G}_q}
\ell(G)
\end{equation}
is not a convex problem because $\mathcal{G}_q$ is nonconvex.  When $q$
is large (and recall that the theory suggests we should take $q=p$),
well-known issues related to nonconvexity and finite mixture models
become a significant obstacle \citep{mclachlan2004finite}.

\subsection{A simple finite-dimensional convex approximation}

In this paper, we take a very simple approach to (approximately)
solving \eqref{KW}, which maintains convexity and
immediately reduces \eqref{KW} to a finite-dimensional problem. Consider a pre-specified finite grid $\Lambda \subseteq \mathcal{T}$.  We study estimators $\hat{G}_{\Lambda}$,
which solve
\begin{equation}\label{KM}
\min_{G \in \mathcal{G}_{\Lambda}} \ell(G).
\end{equation}
The key difference between \eqref{ncvx} and \eqref{KM} is that
$\mathcal{G}_{\Lambda}$, and hence \eqref{KM}, is convex, while $\mathcal{G}_q$ is
nonconvex.  Additionally, \eqref{KM} is a finite-dimensional
optimization problem,
because $\Lambda$ is finite.

To derive a more convenient formulation of \eqref{KM}, suppose that
\begin{equation}\label{Lfinite}
\Lambda = \{t_1,...,t_q\} \subseteq \mathcal{T}
\end{equation}
and define the simplex $\Delta^{q-1} = \{w = (w_1,...,w_q) \in\mathbb{R}^q; \ w_k \geq 0, \ w_1 +
\cdots + w_q = 1\}$.   Additionally, let $\delta_t$ denote a point
mass at $t \in \mathbb{R}^d$.
Then there is a correspondence between $G = \sum_{k = 1}^q w_k\delta_{t_k} \in \mathcal{G}_{\Lambda}$ and points
$w = (w_1,...,w_q) \in \Delta^{q-1}$.  It follows that \eqref{KM} is equivalent to the optimization problem over the
simplex,
\begin{equation}\label{KMsimp}
\min_{w \in \Delta^{q-1}} -\frac{1}{p} \sum_{j = 1}^p
\log\left\{\sum_{k = 1}^q f_0(X_j|t_k)w_k\right\}.
\end{equation}

Researchers studying NPMLEs have previously considered estimators like $\hat{G}_{\Lambda}$, which solve
\eqref{KM}--\eqref{KMsimp}.  However,
most have focused on relatively simple models with univariate mixing distributions $G_0$
\citep{bohning1992computer,jiang2009general,koenker2014convex}.  In very recent work, \cite{gu2017empirical, gu2017unobserved}  have considered multivariate NPMLEs for estimation problems involving Gaussian models --- by contrast, our aim is to formulate strategies for solving and implementing the general problem, as specified in \eqref{KW}, \eqref{KM}, and \eqref{KMsimp}.

\section{Choosing $\Lambda$}
\label{sec:L}

The approximate NPMLE $\hat{G}_{\Lambda}$ is the estimator we use
throughout the rest of the paper.  One remaining question is: How
should $\Lambda$ be chosen?  Our perspective is that $\hat{G}_{\Lambda}$ is an
approximation to $\hat{G}$ and its performance characteristics
are inherited from $\hat{G}$.  In general, $\hat{G}_{\Lambda} \neq \hat{G}$.  However, as
one selects larger and larger finite grids
$\Lambda \subseteq \mathcal{T}$, which are more and more dense in
$\mathcal{T}$, evidently $\hat{G}_{\Lambda} \to
\hat{G}$.  Thus, heuristically, as long as the grid $\Lambda$ is ``dense enough'' in $\mathcal{T}$,
$\hat{G}_{\Lambda}$ should perform similarly to $\hat{G}$.

If $\mathcal{T}$ is compact, then any regular grid $\Lambda \subseteq
\mathcal{T}$ is finite and implementing \eqref{KM} is
straightforward (specific implementations are discussed in Section \ref{sec:implementation}).  Thus, for compact $\mathcal{T}$, one can choose $\Lambda$ to be a
regular grid with as many points as are computationally feasible.  For general $\mathcal{T}$, we propose a
two-step approach to choosing $\Lambda$:  (i) Find a compact convex subset $\mathcal{T}_0
\subseteq \mathcal{T}$ so that \eqref{KW} is equivalent (or
approximately equivalent) to
\begin{equation}\label{KWcpct}
\inf_{G
	\in \mathcal{G}_{\mathcal{T}_0}} \ell(G);
\end{equation}
(ii) choose $\Lambda \subseteq \mathcal{T}_0 \subseteq \mathcal{T}$ to be a
regular grid with $q$
points, for some sufficiently large $q$.  Empirical results
seem to be fairly insensitive to the choice of $q$.  In Sections \ref{sec:bball}--\ref{sec:cgm}, we choose $q = 30^d$ for models with $d=2,3$ dimensional
mixing distributions $G$.  For some simple models with univariate $G$
($d=1$), theoretical results suggest that if $q = \sqrt{p}$, then $\hat{G}_{\Lambda}$ is
statistically indistinguishable from $\hat{G}$
\citep{dicker2016nonparametric}.

Now we form strategies to construct $\Lambda$ for multivariate mixing distribution. First note that \citet{lindsay1981preoperties,lindsay1995mixture} has proposed the construction of $\Lambda$ in one dimensional cases. Our approach may be viewed as a generalization of their work. For each $j =
1,\ldots,p$, define
\[
\hat{\theta}_j =\hat{\theta}(X_j) = \operatorname{argmax} \displaylimits_{\theta \in \mathcal{T}}
f_0(X_j\mid \theta)
\]
to be the maximum likelihood estimator (MLE) for $\Theta_j$, given the data
$X_j \in \mathbb{R}^n$.  The following proposition implies that \eqref{KW} and
\eqref{KWcpct} are equivalent when the likelihoods $f_0(X_j\mid \theta)$
are from a class of elliptical unimodal distributions, and $\mathcal{T}_0 =
\mathrm{conv}(\hat{\theta}_1,\ldots,\hat{\theta}_p)$ is the convex hull of
$\hat{\theta}_1,\ldots, \hat{\theta}_p$.   This result enables us to employ the strategy described above
for choosing $\Lambda$ and finding $\hat{G}_{\Lambda}$; specifically, we take
$\Lambda$ to be a regular grid contained in the compact convex set
$\mathrm{conv}(\hat{\theta}_1,\ldots,\hat{\theta}_p)$.

\begin{prop}\label{prop:conv}
	Suppose that $f_0$ has the form
	\begin{equation}\label{unimodal}
	f_0(X_j \mid \theta) = h\big\{(\hat{\theta}_j -
	\theta)^{\top}\Sigma^{-1}(\hat{\theta}_j - \theta)\big\} u(X_j),
	\end{equation}
	where $h: [0,\infty) \to [0,\infty)$ is a decreasing
	function, $\Sigma$ is a $p \times p$ positive definite matrix, and $u:\mathbb{R}^n
	\to \mathbb{R}$ is some other function that does not depend on $\theta$.  Let $\mathcal{T}_0 =
	\mathrm{conv}(\hat{\theta}_1,\ldots,\hat{\theta}_p)$.  Then
	$\ell(\hat{G}) = \inf_{G \in \mathcal{G}_{\mathcal{T}_0}}
	\ell(G)$.
\end{prop}

\begin{proof}

	Assume that $G = \sum_{k = 1}^q w_k \delta_{t_k}$, where $t_1,\ldots,t_q
	\in \mathcal{T}$ and $w_1,\ldots,w_q > 0$.  Further assume that $t_q
	\notin \mathcal{T}_0= \mathrm{conv}(\hat{\theta}_1,\ldots,\hat{\theta}_p)$.  We show
	that there is another probability distribution $\tilde{G} = \sum_{k =
		1}^{q-1}w_k\delta_{t_k} + w_q\delta_{\tilde{t}_q}$, with $\tilde{t}_q \in
	\mathcal{T}_0$, satisfying $\ell(\tilde{G}) \leq \ell(G)$.  This
	suffices to prove the proposition.

	Let $\tilde{t}_q$ be the projection of $t_q$ onto $\mathcal{T}_0$ with
	respect to the inner product $(s,t) \mapsto s^{\top}\Sigma^{-1}t$.  To
	prove that $\ell(\tilde{G}) \leq \ell(G)$, we show that $f_0(X_j\mid
	\tilde{t}_q) \geq f_0(X_j\mid t_q)$ for each $j = 1,\ldots,p$.  We have
	\begin{align*}
	(\hat{\theta}_j - t_q)^{\top}\Sigma^{-1}(\hat{\theta}_j - t_q) & =(\hat{\theta}_j
	-
	\tilde{t}_q +
	\tilde{t}_q -
	t_q)^{\top}\Sigma^{-1}(\hat{\theta}_j
	-
	\tilde{t}_q +
	\tilde{t}_q -
	t_q)  \\
	& = (\hat{\theta}_j - \tilde{t}_q)^{\top}\Sigma^{-1}(\hat{\theta}_j -
	\tilde{t}_q) + 2(\tilde{t}_q - t_q)^{\top}\Sigma^{-1}(\hat{\theta}_j -
	\tilde{t}_q) \\
	& \qquad + (\tilde{t}_q - t_q)^{\top}\Sigma^{-1}(\tilde{t}_q -
	t_q) \\
	& \geq (\hat{\theta}_j - \tilde{t}_q)^{\top}\Sigma^{-1}(\hat{\theta}_j -
	\tilde{t}_q),
	\end{align*}
	where we have used the fact that $(\tilde{t}_q - t_q)^{\top}\Sigma^{-1}(\hat{\theta}_j -
	\tilde{t}_q) = 0$, because $\tilde{t}_q$ is the projection of
	$t_q$ onto $\mathcal{T}_0$.  By \eqref{unimodal}, it follows that
	$f_0(X_j\mid \tilde{t}_q) \geq f_0(X_j\mid t_q)$, as was to be
	shown.
\end{proof}

The
condition \eqref{unimodal} is rather restrictive, but we believe it
applies in a number of important problems.  The
fundamental example where \eqref{unimodal} holds is $X_j \mid
\Theta_j \sim N(\Theta_j,\Sigma)$;  in
this case $\hat{\theta}_j = X_j$ and \eqref{unimodal} holds with $u$ being certain constant and $h(z) \propto e^{-z/2}$.
Condition \eqref{unimodal} also holds in elliptical models,
where $\Theta_j$ is the location parameter of $X_j\mid \Theta_j$.  More
broadly,  if $X_j = (X_{1j},\ldots,X_{nj})^{\top}  \in \mathbb{R}^n$ may be viewed as a
vector of replicates $X_{ij}$, $i = 1,\ldots,n$, drawn from some common
distribution conditional on $\Theta_j$, then standard results suggest that the MLEs $\hat{\theta}_j$ may be
approximately Gaussian if $n$ is sufficiently large, and
\eqref{unimodal} may be approximately valid.  Specific applications where
a normal approximation argument for $\hat{\theta}_j$ may imply that
\eqref{unimodal} is approximately valid include count data (similar to
Section
\ref{sec:bball}) and time series modeling (Section \ref{sec:cgm}).

\section{Connections with finite mixtures}

Finding $\hat{G}_{\Lambda}$ is equivalent to fitting a
finite-mixture model, where the locations of the atoms for the mixing
measure have been
pre-specified (specifically, the atoms are taken to be the points in
$\Lambda$).  Thus, the approach in this paper reduces computations for the relatively complex
nonparametric mixture model \eqref{MM} to a convex optimization problem that is substantially
simpler than fitting a standard finite mixture model (generally a
non-convex problem).

An important distinction of nonparametric mixture
models is that they lack the built-in interpretability of the
components/atoms from finite mixture models, and are less suited for clustering applications.  On the other hand, taking the
nonparametric approach provides additional flexibility for modeling heterogeneity in applications where it is not clear that
there should be well-defined clusters.  Moreover, post hoc clustering and finite
mixture model methods could still be used after
fitting an NPMLE; this might be advisable if, for instance, $\hat{G}_{\Lambda}$
has several clearly-defined modes.

\section{Implementation overview}
\label{sec:implementation}

A variety of well-known algorithms are available for solving
\eqref{KMsimp} and finding $\hat{G}_{\Lambda}$.  We have experimented with several, including the EM-algorithm, interior point methods, and the Frank-Wolfe algorithm.  This section contains a brief overview of how we have implemented these algorithms; numerical results comparing the algorithms are contained in the following section.

One of the early applications of the EM-algorithm is mixture models \citep{laird1978nonparametric}. Solving \eqref{KMsimp} with the EM-algorithm for mixture models with predetermined grid-points is especially simple, because the problem is convex (recall that the finite mixture model problem --- as opposed to the nonparametric problem --- is typically non-convex).  \cite{koenker2014convex} have developed interior point methods for solving \eqref{KMsimp}.  Along with \cite{koenker2016rebayes}, they created an R package \texttt{REBayes} that solves \eqref{KMsimp} for a handful of specific nonparametric mixture models, e.g. Gaussian mixtures and univariate Poisson mixtures; the \texttt{REBayes} package calls an external optimization software package, Mosek, and relies on Mosek's built-in interior point algorithms.  In our numerical analyses, we used \texttt{REBayes} to compute some one-dimensional NPMLEs with interior point methods.  To estimate multi-dimensional NPMLEs with interior point methods, we used our own implementations based on another R package \texttt{Rmosek} \citep{mosek}  (\texttt{REBayes} does appear to have some built-in functions for estimating two-dimensional NPMLEs, but we found them to be somewhat unstable in our applications).  We note that our interior point implementation solves the primal problem \eqref{KMsimp}, while \texttt{REBayes} solves the dual.  The Frank-Wolfe algorithm \citep{frank1956algorithm} is a classical algorithm for constrained convex optimization problems, which has recently been the subject of renewed attention \cite[e.g.][]{jaggi2013revisiting}.  Our implementation of the Frank-Wolfe algorithm closely resembles the ``vertex direction method,'' which has previously been used for finding the NPMLE in nonparametric mixture models \citep{bohning1995review}.

All of the algorithms used in this paper were implemented in R.  We did not attempt to heavily optimize any of these implementations; instead, our main objective was to demonstrate that there are a range of simple and effective methods for finding (approximate) NPMLEs.  While the \texttt{REBayes} and \texttt{Rmosek} packages were used for their interior point methods, no packages beyond base R were required for any of our other implementations.

\section{Simulation studies}

This section contains simulation results for NPMLEs and a Gaussian location-scale mixture model.  Section \ref{sec:numAlg} contains a comparison of the various NPMLE algorithms described in the previous section.  In Section \ref{sec:numMean}, we compare the performance of NPMLE-based estimators to other commonly used methods for estimating the mean in a Gaussian location-scale model.

In all of the simulations described in this section we generated the data as follows.  For $j = 1,\ldots,p$, we generated independent $\Theta_j = (\mu_j,\sigma_j) \sim G_0$ and corresponding observations $X_j \in \mathbb{R}^n$.  Each $X_j  = (X_{1j},\ldots,X_{nj})^{\top} $ was a vector of $n$ replicates
$X_{1j},\ldots,X_{nj}\mid\Theta_j \sim N(\mu_j,\sigma_j^2)$ that were generated independently, conditional on $\Theta_j$. In other words,
\begin{equation}\label{GLS}
X_{j}\mid\Theta_j \sim f_0(X_j|\Theta_j)= \left(\frac{1}{\sqrt{2\pi}\sigma_j}\right)^n\exp\left\{-\frac{\sum_{i=1}^n(X_{ij}-\mu_j)^2}{2\sigma_j^2}\right\}.
\end{equation}
In the general model \eqref{GLS}, the mixing distribution $G_0$ is bivariate.  However, we considered two values of $G_0$ in our simulations: one where the marginal distribution of $\sigma_j$ was degenerate (i.e. $\sigma_j$ was constant, so $G_0$ was effectively univariate) and one  where the marginal distribution of $\sigma_j$ was non-degenerate.  (Note that for non-degenerate $\sigma_j$, $n \geq 2$ replicates are essential in order to ensure that the likelihood is bounded and that $\hat{G}$ exists.)

Throughout the simulations, we took $p = 1000$ and $n = 16$.  For the first mixing distribution $G_0$ (degenerate $\sigma_j$), we fixed $\sigma_j = 4$ and took $\mu_j$ so that ${\mathbb{P}}(\mu_j = 0) = {\mathbb{P}}(\mu_j = 5) = 1/2$.  For the second mixing distribution (non-degenerate $\sigma_j$), we took ${\mathbb{P}}(\mu_j = 0, \ \sigma_j = 5) = {\mathbb{P}}(\mu_j = 5, \ \sigma_j = 3) = 0.5$; for this distribution $\mu_j$ and $\sigma_j$ are correlated.

\subsection{Comparing NPMLE algorithms}\label{sec:numAlg}

For each mixing distribution, we computed $\hat{G}_{\Lambda}$ using the algorithms described in Section \ref{sec:implementation}: the EM-algorithm, an interior point method with \texttt{Rmosek}, and the Frank-Wolfe algorithm.  For each of these algorithms, we also computed $\hat{G}_{\Lambda}$ for various grids $\Lambda$.  Specifically, we considered regular grids $\Lambda = \{m_k\}_{k= 1}^{q_1} \times \{s_k\}_{k = 1}^{q_2}  \subseteq [\min \hat{\mu}_j, \max\hat{\mu_j}] \times [\min \hat{\sigma}_j, \max\hat{\sigma}_j]  \subseteq \mathbb{R}^2$, where $\hat{\mu}_j = n^{-1}\sum_i X_{ij}$ and $\hat{\sigma}_j^2 = n^{-1}\sum_i (X_{ij} - \hat{\mu}_j)^2$.  The values $q_1,q_2$ determine number of grid-points in $\Lambda$ for $\mu_j,\sigma_j$, respectively, and in the simulations we fit estimators with $(q_1,q_2) = (30,30), (50,50)$, and $(100,100)$. The stopping criterion of EM and Frank-Wolfe is
		\begin{eqnarray*}
		|\ell(w^{(K)})-\ell(w^{K-1})|/|\ell(w^{(K-1)}| \le 10^{-6},
		\end{eqnarray*}
where $\ell(w)$ is the objective function in (\ref{KMsimp}) and $w^{(K)}$ is the solution at the $K$-th iteration of respective methods.

In addition to fitting the two-dimensional NPMLEs $\hat{G}_{\Lambda}$ described above, for the simulations with degenerate $\sigma_j$ we also fit one-dimensional NPMLEs to the data $\hat{\mu}_1,\ldots,\hat{\mu}_p$, according to the model
\[
\hat{\mu}_j \mid \mu_j \sim N(\mu_j,1).
\]
We fit these one-dimensional NPMLEs using all of the same algorithms for the two-dimensional NPMLEs (EM, interior point with \texttt{Rmosek}, and Frank-Wolfe), and we also used the \texttt{REBayes} interior point implementation to estimate the distribution of $\mu_j$ in this setting.  For the one-dimensional NPMLEs, we took $\Lambda \subseteq [\min \hat{\mu}_j,\max\hat{\mu}_j] \subseteq \mathbb{R}$ to be the regular grid with $q = 300$ points.  This allows us to compare the performance of methods for one- and two-dimensional NPLMEs (where the one-dimensional NPMLEs take the distribution of $\sigma_j$ to be known) and compare the performance of the two interior point algorithms, among other things.

For each simulated dataset and estimator $\hat{G}_{\Lambda}$ we recorded several metrics.  First, we computed the total squared error (TSE)
	\begin{eqnarray*}
	\mathrm{TSE} = \sum_{j = 1}^p (\muhat_j - \mu_j)^2,
	\end{eqnarray*}
	where
	\begin{eqnarray*}
	\hat{\mu}_j = \mathbb{E}_{\hat{G}_{\Lambda}}(\mu_j \mid X_j)=\frac{\int\mu_jf_0(X_j|\mu_j,\sigma_j)d\hat{G}_{\Lambda}(\mu_j,\sigma_j)}{\int f_0(X_j|\mu_j,\sigma_j)d\hat{G}_{\Lambda}(\mu_j,\sigma_j)}.
	\end{eqnarray*}
Second, we computed the difference between the log-likelihood of $\hat{G}_{\Lambda}$ and the log-likelihood of $\hat{G}_{\mathrm{EM}}$, the corresponding estimator for $G_0$ based on the EM-algorithm:
\[
\Delta(\mbox{log-lik.}) = \ell(\hat{G}_{\mathrm{EM}}) - \ell(\hat{G}_{\Lambda}).
\]
Note that $\Delta(\mbox{log-lik.}) > 0$ if $\hat{G}_{\Lambda}$ has a {\em smaller} negative log-likelihood than $\hat{G}_{\mathrm{EM}}$ (we are taking the EM-estimator $\hat{G}_{\mathrm{EM}}$ as a baseline for measuring the log-likelihood). Finally, we recorded the time required to compute $\hat{G}_{\Lambda}$ (in seconds; all calculations were performed on a 2015 MacBook Pro laptop).  Summary statistics are reported in Table \ref{tab:alg}.

It is evident that the results in Table \ref{tab:alg} are relatively insensitive to the number of grid-points $(q_1,q_2)$ chosen for the two-dimensional NPMLE implementations.  In terms of TSE, the EM algorithm and interior point methods perform very similarly across all of the settings, while the interior point methods appear to slightly out-perform the EM algorithm in terms of $\Delta(\mbox{log-lik.})$ across the board.  Additionally, the interior point methods have smaller compute time than the EM algorithm, though the difference is not too significant for applications at this scale (for mixing distribution 1, with degenerate $\sigma_j$, the \texttt{REBayes} dual implementation appears to be somewhat faster than our \texttt{Rmosek} primal implementation).  The Frank-Wolfe algorithm is the fastest implementation we have considered, but its performance in terms of TSE and $\Delta(\mbox{log-lik.})$ is considerably worse than the EM algorithm or interior point methods.  In the remainder of the paper, we chose to use the EM algorithm exclusively for computing NPMLEs --- we believe it strikes a balance between simplicity and performance.

\begin{table}[H]
	\begin{center}
		\caption{Comparison of different NPMLE algorithms.  Mean values (standard deviation in parentheses) reported from 100 independent datasets; $p=1000$, throughout simulations. Mixing distribution 1 has constant $\sigma_j$; mixing distribution 2 has correlated $\mu_j$ and $\sigma_j$.}
		\label{tab:alg}
		\setlength{\tabcolsep}{1pt}
		\begin{tabular}{clccc}
			&& TSE &\begin{tabular}[b]{c} $\Delta(\text{log-lik})$\\ $\! \times\! 10^{4}$\end{tabular} &\begin{tabular}[b]{c} Time\\ (secs.) \end{tabular}\\ \hline
			Mixing dist. 1&EM &&& \\
			(Bivariate)&\ \ \ $(q_1,q_2) = (30,30)$ &130.5  (42.6) & 0 (0)& 9\\
			&\ \ \  $(q_1,q_2) = (50,50)$& 130.4 (42.7)& 0 (0)& 33\\
			&\ \ \  $(q_1,q_2) = (100,100)$&130.4 (42.6) &0 (0)& 136
			\\
			&Interior point (\texttt{Rmosek}) &&&\\
			&\ \ \ $(q_1,q_2) = (30,30)$&130.7 (42.6)& 6 (1)& 8\\
			&\ \ \ $(q_1,q_2) = (50,50)$& 130.5 (42.9)& 9 (1)&20 \\
			&\ \  \ $(q_1,q_2) = (100,100)$&130.6 (42.8) & 11 (1)  &80 \\
			&Frank-Wolfe  &&& \\
			&\ \ \ $(q_1,q_2) =
			(30,30)$&147.3 (45.1)& -234 (130)& 5 \\
			&\ \ \   $(q_1,q_2) = (50,50)$& 147.0 (45.9)& -238 (134)& 14\\
			&\ \ \  $(q_1,q_2) = (100,100)$&146.2 (45.4)& -238 (128)& 55\\\hline
			Mixing dist. 1&EM&124.4 (41.6) & 0 (0)& 1
			\\
			(univariate; $q = 300$;	&Interior point (\texttt{Rmosek}) &124.3 (42.1)& 6 (1)&3\\
			assume known $\sigma_j$)&Interior point (\texttt{REBayes})&124.3 (42.1)& 6 (1) &1  \\
			&	Frank-Wolfe & 126.1 (41.8)& -4 (4)& 1 \\
			\hline
			Mixing dist. 2&		EM &&& \\
			&	\ \ \ $(q_1,q_2) = (30,30)$& 54.0 (28.4) &0 (0)& 9\\
			&\ \ \  $(q_1,q_2) = (50,50)$&  54.0 (28.9) & 0 (0) &34\\
			&		\ \ \  $(q_1,q_2) = (100,100)$&53.9 (28.8) &0 (0) &141 \\
			&	Interior point (\texttt{Rmosek}) &&& \\
			&	\ \ \ $(q_1,q_2) = (30,30)$& 54.3 (28.8) & 5 (1) &8\\
			&	\ \ \ $(q_1,q_2) = (50,50)$&54.2 (29.1)& 8 (1)&20\\
			&	\ \  \ $(q_1,q_2) = (100,100)$ &54.3 (29.1) & 10 (1) &82 \\
			&	Frank-Wolfe  &&& \\
			&	\ \ \ $(q_1,q_2) =
			(30,30)$& 82.2 (39.3)& -372 (217) &5 \\
			&	\ \ \   $(q_1,q_2) = (50,50)$&83.1 (36.4) & -402 (232) &14\\
			&	\ \ \  $(q_1,q_2) = (100,100)$&82.0 (37.7) & -396 (240) & 56\\
			\hline
		\end{tabular}
	\end{center}
\end{table}

\subsection{Gaussian location-scale mixtures: Other methods for estimating a normal mean vector} \label{sec:numMean}

Beyond NPMLEs, we also implemented several other methods that are commonly used for estimating the mean vector $\boldsymbol{\mu} = (\mu_1,\ldots,\mu_p)^\top \in \mathbb{R}^p$ in Gaussian location-scale models and computed the corresponding TSE.  Specifically, we considered the fixed-$\Theta_j$ MLE, $\boldsymbol{\muhat} = (\hat{\mu}_1,\ldots,\hat{\mu}_p)^\top \in \mathbb{R}^p$; the James-Stein estimator; the heteroscedastic SURE estimator of \citet{xie2012sure}; and a soft-thresholding estimator. The James-Stein estimator is a classical shrinkage estimator for the
Gaussian location model.  The version employed here is described in \cite{xie2012sure} and is designed for
heteroscedastic data. The heteroscedastic SURE estimator is another shrinkage estimator,
which was designed to ensure that features with a high noise variance
are ``shrunk'' more than those with a low noise variance. Both the James-Stein estimator and the heteroscedastic SURE estimator depend on the values $\sigma_j$.
The soft-thresholding
estimator takes the form $\hat{\mu}(X) = s_t(\hat{\mu}_j)$, where $t \geq  0$ is a constant and $s_t(x) = \mathrm{sign}(x)\max\{|x| - t,0\}$,
$x \in \mathbb{R}$.  For soft-thresholding estimators, $t$ was chosen to minimize the TSE.  Observe that the James-Stein, SURE, and soft-thresholding estimators all depend on information that is typically not available in practice: the value of $\sigma_j$ and the actual TSE.  By contrast, the two-dimensional NPMLEs described in the previous sub-section utilize only the observed data $X_1,\ldots,X_p$.

In Table \ref{tab:est}, we report the TSE for the different estimators described in this section, along with the TSE for the bivariate NPMLE fit using the EM algorithm.  We also fit a univariate NPMLE in this example, where $\sigma_j$ was {\em not} assumed to be known; instead we used the plug-in estimator $\hat{\sigma}_j$ in place of $\sigma_j$ and then computed the NPMLE for the distribution of $\mu_j$.

Table \ref{tab:est} shows that the NPMLEs dramatically out-perform the alternative estimators in this setting, in terms of TSE. The bivariate NPMLE out-performs the univariate NPMLE under both mixing distributions 1 and 2, but its advantage is especially pronounced under mixing distribution 2, where $\mu_j$ and $\sigma_j$ are correlated.  This highlights the potential advantages of bivariate NPMLEs over univariate approaches in settings with multiple parameters.

\begin{table}[h]
	\begin{center}
		\caption{Mean TSE for various estimators of $\boldsymbol{\mu} \in \mathbb{R}^p$ based on 100 simulated datasets; $p = 1000$. $(q_1,q_2)$ indicates the grid-points used to fit $\hat{G}_{\Lambda}$.}
		\label{tab:est}
		\setlength{\tabcolsep}{1.2pt}
		\begin{tabular}{lccclccc}
			Method & Mixing dist.1 & \ Mixing dist.2 \\ \hline
			Fixed-$\Theta_j$ MLE & 997.0 (48.2)&1059.3 (57.9)\\
			Soft-Thresholding &826.2 (50.0)& 793.7 (46.0) \\
			James-Stein & 859.7 (43.6)&935.2 (53.0)\\
			SURE &859.7 (43.6)& 880.7 (48.1)\\
			Univariate NPMLE $q = 300$ & 170.7 (47.0) & 285.4 (63.5)\\
			Bivariate NPMLE $(q_1,q_2)=(100, 100)$ &130.4 (42.6)&53.9 (28.8)
		\end{tabular}
	\end{center}
\end{table}

\section{Baseball data}
\label{sec:bball}

Baseball data is a well-established testing
ground for empirical Bayes methods \citep{efron1975data}.  The
baseball dataset we analyzed contains the number of at-bats and
hits for all of the Major League Baseball players during the 2005
season and has been previously analyzed in a number of papers \citep{brown2008season,jiang2010empirical,muralidharan2010empirical,xie2012sure}.
The goal of the analysis is to use the data from the first half of the
season to predict each player's batting average (hits/at-bats) during
the second half of the season.  Overall, there are 929 players in the
baseball dataset; however, following \cite{brown2008season} and others, we
restrict attention to the 567 players with more than 10
at-bats during the first half of the season for training and among which 499 players with more than 10 at-bats during the second half of season for prediction. (We follow the other
preprocessing steps described in \cite{brown2008season} as well).

Let $A_{j}$ and $H_{j}$ denote the number
of at-bats and hits, respectively, for player $j$ during the first
half of the season, $j=1,...,567$.  We assume that $(A_j,H_j)$ follows a
Poisson-binomial mixture model, where $A_j\mid (\lambda_j,\pi_j) \sim \mbox{Poisson}(\lambda_j)$,  $H_j\mid
(A_j,\lambda_j,\pi_j) \sim \mbox{binomial}(A_j,\pi_j)$,
and $(\lambda_j,\pi_j) \sim G_0$.  This model has a bivariate mixing distribution $G_0$, i.e. $d = 2$.  In the notation of \eqref{MM}, $X_j =
(A_j,H_j)$, $\Theta_j = (\lambda_j,\pi_j)$ and
\begin{eqnarray*}
f_0(X_j|\Theta_j)=f_0(A_j, H_j|\lambda_j,\pi_j)=\frac{\lambda_j^{A_j}e^{-\lambda_j}}{A_j!}\times {{A_j}\choose{H_j}}\pi_j^{H_j}(1-\pi_j)^{A_j-H_j}.
\end{eqnarray*}
We took $\Lambda\subseteq [\min A_j, \max A_j]\times [ \min A_j/H_j, \max A_j/H_j]$ to be the regular grid with $(q_1,q_2)=(30,30)$.
We propose
to estimate each
player's batting average for the second half of the season by
the posterior mean of $\pi$, computed under $(\lambda,\pi) \sim
\hat{G}_{\Lambda}$,
\begin{equation}\label{pi_hat}
\hat{\pi}_j = \mathbb{E}_{\hat{G}_{\Lambda}}(\pi_j \mid A_j,H_j)=\frac{\int\pi_jf_0(A_j, H_j|\lambda_j,\pi_j)d\hat{G}_{\Lambda}(\lambda_j,\pi_j)}{\int f_0(A_j, H_j|\lambda_j,\pi_j)d\hat{G}_{\Lambda}(\lambda_j,\pi_j)}.
\end{equation}

Most previously published analyses of the baseball data begin by transforming
the data via the variance stabilizing transformation
\begin{equation}\label{stab}
W_j = \arcsin
\sqrt{\frac{H_j + 1/4}{A_j + 1/2}}
\end{equation}
(\citet{muralidharan2010empirical} is a notable exception).
Under this transformation,
$W_j$ is approximately distributed as $N\{\mu_j,(4A_j)^{-1}\}$, where $\mu_j = \arcsin
\sqrt{\pi_j}$. Methods for Gaussian observations may be applied to the transformed
data, with the objective of estimating $\mu_j$.  Following this approach, a variety of methods based on shrinkage,
the James-Stein estimator, and parametric empirical
Bayes methods for Gaussian data
have been
proposed and studied \citep{brown2008season,jiang2010empirical,xie2012sure}.

Under the transformation \eqref{stab}, it is standard to use total squared error to measure
the performance of estimators $\hat{\mu}_j$ \citep[e.g.][]{brown2008season}.  In this example, the total squared error is defined as
\[
\mathrm{TSE} = \sum_{j=1,...,499} \left\{(\hat{\mu}_j - \tilde{W}_j)^2 -
\frac{1}{4\tilde{A}_j}\right\},
\]
where
\[
\tilde{W}_j = \arcsin \sqrt{\frac{\tilde{H}_j +
		1/4}{\tilde{A}_j + 1/2}},
\] and $\tilde{A}_j$ and $\tilde{H}_j$ denote the
at-bats and hits from the second half of the season, respectively.
For convenience of comparison, we used $\mathrm{TSE}$ to measure
the performance of our estimates $\hat{\pi}_j$, after applying the
transformation $\hat{\mu}_j = \arcsin \sqrt{\hat{\pi}_j}$.

Results from the baseball analysis are reported in
Table \ref{tab:bball}.  Following the work of others, we have analyzed
all players from the dataset together, and then the pitchers and
non-pitchers from the dataset separately.
 In addition to our
Poisson-binomial NPMLE-based estimators \eqref{pi_hat}, we
considered six other previously studied estimators:
\begin{enumerate}
	\item  The (fixed-parameter) {\em MLE}
	estimator $\hat{\mu}_j = W_j$ uses each player's hits and at-bats from the first half of
	the season to estimate their performance in the second half.
	\item The
	{\em grand mean} $\hat{\mu}_j = p^{-1}(W_1 + \cdots + W_p)$ gives the
	exact same estimate for each player's performance in the second half of the season,
	which is equal to the average performance of all players during the
	first half.
	\item The {\em James-Stein} parametric empirical Bayes estimator described in
	\cite{brown2008season}.
	\item The weighted generalized MLE ({\em weighted GMLE}), which uses at-bats as a
	covariate \citep{jiang2010empirical}.  This is essentially a
	univariate NPMLE-method for Gaussian models with covariates.
	\item The {\em semiparametric SURE} estimator is a flexible shrinkage estimator that may be viewed as a
	generalization of the James-Stein estimator \citep{xie2012sure}.
	\item The {\em binomial mixture} method in
	\cite{muralidharan2010empirical} is another empirical Bayes method, which does not require the data to be
	transformed and estimates $\pi_j$ directly (in \cite{muralidharan2010empirical}, they work conditionally on
	the at-bats $A_j$).  $\mathrm{TSE}$ is computed after applying the $\mathrm{arcsin}\sqrt{\cdot}$ transformation.
\end{enumerate}

\begin{table}[h]
	\begin{center}
		\caption{Baseball data.  $\mathrm{TSE}$ relative to the
			MLE. Minimum error is in {\bf bold} for each analysis.}
		\label{tab:bball}
		\setlength{\tabcolsep}{1pt}
		\begin{tabular}{lccclccc}
			& &Non- & &&& Non- & \\ Method &All&Pitchers&
			Pitchers
			& Method & All & Pitchers & Pitchers  \\ \hline
			 MLE& 1&1&1 & SURE  & 0.41 &
			{\bf 0.26}
			& {\bf 0.08} \\
			Grand mean & 0.85& 0.38& 0.13 & Binomial
			mixture & 0.59
			& 0.31 & 0.16 \\
			James-Stein &0.54& 0.35&0.17 &
			NPMLE
			& {\bf
				0.29} &
			{\bf 0.26} & 0.14 \\
			GMLE &0.30& {\bf 0.26}&0.14
		\end{tabular}
	\end{center}
\end{table}

The values reported in Table \ref{tab:bball} are the $\mathrm{TSE}$s of each
estimator,
relative to the $\mathrm{TSE}$ of the fixed-parameter MLE.  Our
Poisson-binomial method performs very well, recording the minimum
$\mathrm{TSE}$ when all of the data (pitchers and non-pitchers) are
analyzed together and
for the non-pitchers.  Moreover, the Poisson-binomial NPMLE
$\hat{G}_{\Lambda}$ works on the original scale of the data (no
angular transformation is required) and may be
useful for other purposes, beyond just estimation/prediction. Figure
\ref{fig:bball_dist} (a) plots the marginal Poisson-binomial NPMLE distribution of $\pi$, estimated with EM algorithm using all players, non-pitchers and pitchers data. Figure
\ref{fig:bball_dist} (b) contains a histogram and contours of 20,000 independent draws from the
estimated mixture distribution of $(A_j,H_j/A_j)$, fitted with the
Poisson-binomial NPMLE to all players in the baseball dataset.  Observe that
the distribution appears to be bimodal.   By comparing
this histogram with histograms of the observed data from the
non-pitchers and pitchers separately (Figure \ref{fig:bball_dist} (c)--(d)),
it appears that the mode at the left of Figure \ref{fig:bball_dist} (b)
represents a group of players that includes the pitchers and the mode at the right
represents the bulk of the non-pitchers.

\begin{figure*}[t]
	\centering
	\includegraphics[width=\textwidth]{fig_revised.pdf}
	\caption{\label{fig:bball_dist} (a) Marginal NPMLE distribution of $\pi$, computed using EM algorithm; (b) histogram of 20,000 independent
		draws from the estimated distribution of $(A_j,H_j/A_j)$, fitted with the
		Poisson-binomial NPMLE to all players in the baseball dataset; (c)
		histogram of non-pitcher
		data from the baseball dataset; (d) histogram of pitcher data from the baseball
		dataset.}
\end{figure*}

\section{Two-dimensional NPMLE for cancer microarray classification}
\label{sec:class}

\cite{dicker2016nonparametric} proposed a univariate NPMLE-based method for
high-dimensional classification problems and studied  applications
involving cancer microarray data.  The classifiers from
\cite{dicker2016nonparametric} are based on a Gaussian model
with one-dimensional mixing
distributions, i.e. $d=1$.  In this section we show that using a bivariate
mixing distribution may substantially improve performance.

Two datasets from the Microarray
Quality Control Phase II project \citep{maqc2010microarray} are considered; one
from a breast cancer study and one from a myeloma study.  The
training dataset for the breast cancer study contains $n = 130$
subjects and $p = 22283$ probesets (genes); the test dataset
contains $100$ subjects.  The training dataset for
the myeloma study contains $n = 340$ subjects and $p = 54675$
probesets; the test dataset contains $214$ subjects.   The goal
is to use the training data to build binary classifiers for several
outcomes, then check the
performance of these classifiers on the test data.   Outcomes for
the breast cancer data are response to treatment (``Response'') and
estrogen receptor status (``ER status''); outcomes for the myeloma
data are overall and event-free survival (``OS'' and ``EFS'').

For each
of the studies, let
$X_{ij}$ denote the expression level of gene $j$ in subject $i$ and
let $Y_i \in \{0,1\}$ be the class label for subject $i$.  Let $X_j =
(X_{1j},\ldots,X_{nj}) ^{\top} \in \mathbb{R}^n$.  We assume that each class ($k
= 0,1$) and
each gene ($j=1,\ldots,p$) has an associated mean expression level
$\mu_{jk} \in \mathbb{R}$, and that conditional on the $Y_i$ and $\mu_{jk}$ all of
the $X_{ij}$ are independent and Gaussian, satisfying $X_{ij}\mid (Y_i = k, \ \mu_{jk}) \sim
N(\mu_{jk},1)$  (the gene-expression levels in the datasets are all
standardized to have variance 1).

In \cite{dicker2016nonparametric}, they assume
that $\mu_{1k},\ldots,\mu_{pk} \sim G_k$ ($k = 0,1$) are all independent draws
from two distributions, $G_0$ and $G_1$.  They use the
training data from classes $k= 0,1$ to separately estimate the
distributions $G_0$ and $G_1$ using NPMLEs, and then implement the Bayes
classifier, replacing $G_0$ and $G_1$ with the corresponding
estimates.   In this paper, we model $\Theta_j = (\mu_{j0},\mu_{j1})
\sim G_0$ jointly, then compute the bivariate
NPMLE $\hat{G}_{\Lambda}$, and finally use $\hat{G}_{\Lambda}$ in place of $G_0$
in the Bayes classifier for this model.
In the notation of \eqref{MM}, $\Theta_j = (\mu_{j0},\mu_{j1})$ and
	\begin{eqnarray*}
	X_j\mid\Theta_j \sim f_0(X_j|\Theta_j)\propto\exp\left[-\frac{1}{2}\sum_{i=1}^n\left\{X_{ij}-\sum_{k=0,1}\mu_{jk}\mathbbm{1}(Y_i=k)\right\}^2\right],
	\end{eqnarray*}
where $\mathbbm{1}(\cdot)$ is the indicator function. The model from
\cite{dicker2016nonparametric} is equivalent to the model proposed
here, when $\mu_{j0}$ and $\mu_{j1}$ are independent.   Results
from analyzing the MAQC datasets using these two
classifiers (the previously proposed method with 1-dimensional
NPMLEs and the 2-dimensional NPMLE described here), along with some
other well-known and relevant classifiers, may be found in
Table \ref{tab:class}. The other classifiers we considered were:

\begin{enumerate}
	\item {\em NP EBayes w/smoothing}.  Another nonparametric empirical
	Bayes classifier proposed in \cite{greenshtein2009application}, which
	uses nonparametric smoothing to fit a univariate density to the
	$\mu_{jk}$

	and then employs a version of linear discriminant analysis.
	\item {\em  Regularized LDA}. A version of $\ell^1$-regularized linear discriminant analysis, proposed in
	\citet{mai2012direct}.
	\item {\em Logistic lasso}. $\ell^1$-penalized logistic regression fit
	using the R package \texttt{glmnet}.
\end{enumerate}

\begin{table}[t]
	\begin{center}
		\caption{Microarray data.  Number of misclassification
			errors on test data. }
		\label{tab:class}
		\setlength{\tabcolsep}{3pt}
		\begin{tabular}{llccccccc} &&&&&& &Logistic & \\
			Dataset&
			Outcome& $n_{\mathrm{test}}$
			&2d-NPMLE&1d-NPMLE& EBayes
			& LDA
			&lasso
			\\ \hline
			Breast&Response& 100 &{\bf 15}&36&47&30&18\\
			Breast&ER status& 100 &19&40&39&{\bf 11} & {\bf 11}\\
			Myeloma&OS& 214 &30&55&100&97&{\bf 27}\\
			Myeloma&EFS& 214&34&76&100&63&{\bf 32}
		\end{tabular}
	\end{center}
\end{table}

For each of the datasets and outcomes, the 2-dimensional NPMLE
classifier substantially outperforms the 1-dimensional NPMLE, and is very
competitive with the top performing classifiers.  Modeling dependence
between $\mu_{j0}$ and $\mu_{j1}$, as with the 2-dimensional NPMLE,
seems sensible because most of the genes are likely to have similar
expression levels across classes, i.e. $\mu_{j0}$ and $\mu_{j1}$ are
likely to be correlated.  This may be interpreted as a kind of sparsity
assumption on the data, which is prevalent in high-dimensional classification
problems.  Moreover, our proposed method involving NPMLEs
should adapt to non-sparse settings as well, since $G_0$ is allowed to
be an arbitrary bivariate distribution.

One of the main underlying assumptions of the NPMLE-based
classification methods is that the different genes have independent
expression levels.  This is certainly {\em not} true in most applications,
but is similar in principle to a ``naive Bayes'' assumption.
Developing methods for NPMLE-based classifiers to better handle correlation
in the data may be of interest for future research.

\section{Continuous glucose monitoring}
\label{sec:cgm}

The analysis in this section is based on blood glucose data from a
study involving 137 type 1 diabetes patients; more details on the study may be
found in \cite{hirsch2008sensor} and \cite{dicker2013continuous}.  Subjects in
the study were monitored for an average of 6 months each.  Throughout
the course of the
study, each subject wore a continuous glucose monitoring device, built around
an electrochemical glucose biosensor.  Every 5 minutes while in use,
the device records (i) a raw electrical current measurement from the
sensor (denoted $\mathrm{ISIG}$), which is
known to be correlated with blood glucose density, and (ii) a timestamped
estimate of blood glucose density ($\mathrm{CGM}$), which is based on a proprietary algorithm
for converting the available data (including the electrical current
measurements from the sensor) into blood glucose density estimates.
In addition to using the sensors, each study subject maintained a
self-monitoring routine, whereby blood glucose density was measured
approximately 4 times per day from a small blood sample extracted by
fingerstick.   Fingerstick measurements of blood glucose density are considered to be more
accurate (and are more invasive) than the sensor-based estimates
(e.g. CGM).
During the study, the
result of each fingerstick measurement was manually entered into the
continuous monitoring device at the time of measurement; algorithms
for deriving continuous sensor-based estimates of blood glucose
density, such as $\mathrm{CGM}$, may use the available fingerstick
measurements for calibration purposes.

In the rest of this section, we show how NPMLE-based empirical
Bayes methods can be used to improve algorithms for estimating blood glucose
density using the continuous monitoring data.  The basic idea is that
after formulating a statistical model relating blood glucose density
to $\mathrm{ISIG}$, we allow for the possibility that the model parameters may
differ for each subject, then use a training dataset to estimate the
distribution of model parameters across subjects (i.e. estimate
$G_0$) via nonparametric maximum likelihood.  This is illustrated for
two different statistical models in Sections \ref{sec:LM}--\ref{sec:KF}.

Throughout the analysis below, we use fingerstick measurements as a proxy
for the actual blood glucose density values.  Let $\mathrm{FS}_j(t)$ and
$\mathrm{ISIG}_j(t)$ denote the fingerstick blood glucose density and $\mathrm{ISIG}$
values, respectively, for the $j$-th subject at time $t$.   Recall
that $\mathrm{FS}_j(t)$ is measured, on average, once every 6
hours, while $\mathrm{ISIG}_j(t)$ is available every 5 minutes.  Let
$\mathcal{F}_t$ denote the $\sigma$-field of information available at time $t$
(i.e. all of the fingerstick and $\mathrm{ISIG}$ measurements taken before
time $t$, plus $\mathrm{ISIG}_j(t)$).  For each methodology, we use the first half of the available
data for each subject to fit a statistical model relating $\mathrm{ISIG}_j(t)$
to $\mathrm{FS}_j(t)$, then estimate each value $\mathrm{FS}_j(t)$ in the second half
of the data using $\widehat{\mathrm{FS}}_j(t)$, an estimator based on $\mathcal{F}_t$.
The performance of each method is measured by the MSE
on the test data, relative to the MSE of the proprietary estimator
$\mathrm{CGM}$.

\subsection{Linear model}\label{sec:LM}

First we consider a basic linear regression model relating $\mathrm{FS}$ and $\mathrm{ISIG}$,
\begin{equation}\label{LM}
\mathrm{FS}_j(t) = \mu_j + \beta_j \mathrm{ISIG}_j(t) + \sigma_j \epsilon_j(t),  \ \ t=t_1,...,t_n
\end{equation}
where the $\epsilon_j(t)$ are iid $N(0,1)$, $t_1,...,t_n$ is the FS measurement time and $\Theta_j
= \{\mu_j,\beta_j,\log(\sigma_j)\} \in \mathbb{R}^3$ are unknown, subject-specific
parameters. In the notation of \eqref{MM}, $X_j =\mathrm{FS}_j\equiv\{\mathrm{FS}_j(t), t=t_1,..,t_n\}$ and
	\begin{eqnarray*}
	X_j\mid\Theta_j &\sim&  f_0(\mathrm{FS}_j|\Theta_j)\cr&\propto&\exp\left[-n\log\sigma_j-\frac{\sum_{i=1}^n\big\{\mathrm{FS}_j(t_i)-\mu_j- \beta_j \mathrm{ISIG}_j(t_i)\big\}^2}{2\exp(2\log\sigma_j)}\right].
	\end{eqnarray*}
Three ways to fit \eqref{LM} are (i) using the combined model,
where $\Theta_j = \Theta = \{\mu,\beta,\log(\sigma)\}$ for all $j$, i.e. all the
subject-specific parameters are the same; (ii) the individual model,
where $\Theta_1,\ldots,\Theta_p$ are all estimated separately, from the
corresponding subject data; and (iii) the nonparametric mixture model, where $\Theta_j
\sim G_0$ are iid draws from the $d=3$-dimensional mixing distribution
$G_0$.
 For each of these methods, we took $\widehat{\mathrm{FS}}_j(t) =
\hat{\mu}_j + \hat{\beta}_j \mathrm{ISIG}(t)$, where $\hat{\mu}_j$
and $\hat{\beta}_j$ are the corresponding MLEs under the combined and
individual models, and, under the mixture model, $\hat{\mu}_j =
\mathbb{E}_{\hat{G}_{\Lambda}}(\mu_j\mid \mathcal{F}_t)$ and $\hat{\beta}_j =
\mathbb{E}_{\hat{G}_{\Lambda}}(\beta_j\mid \mathcal{F}_t)$.  Results are reported in Table
\ref{tab:BG}.

\begin{table}[t]
	\begin{center}
		\caption{Blood glucose data. MSE relative to CGM.}\label{tab:BG}
		\begin{tabular}{cccccc}
			\multicolumn{3}{c}{Linear model} &
			\multicolumn{3}{c}{Kalman filter} \\
			Combined & Individual & NPMLE & Combined & Individual & NPMLE
			\\ \hline 1.56 & 1.54 & {\bf 1.51} & 1.05 & 1.07 &
			{\bf 1.03}
		\end{tabular}
	\end{center}
\end{table}

\subsection{Kalman filter}\label{sec:KF}

Substantial performance improvements are possible by allowing the
model parameters relating $\mathrm{FS}$ and $\mathrm{ISIG}$ to vary with time.  In
this section we consider the Gaussian state space model (Kalman filter)
\begin{equation}\label{KF}
\begin{array}{rcl}
\mathrm{FS}_j(t_i) \!\!\!\!& = &\!\!\!\! \alpha_j(t_i)\mathrm{ISIG}_j(t_i) + \sigma_j\epsilon_j(t_{i-1}), \\
\alpha_j(t_i) \!\!\!\!& = & \!\!\!\!\alpha_j(t_{i-1}) + \tau_j
\delta_j(t_{i-1}),
\end{array}
\end{equation}
where we assume that $\mathrm{FS}$ along with $\mathrm{ISIG}$ are observed at times $t_1,\ldots,t_n$ and $\epsilon_j(t),\delta_j(t) \sim N(0,1)$ are iid.  In \eqref{KF},
$\{\alpha_j(t)\}$ are the state variables that evolve according to a
random walk and $\Theta_j = \{\log(\tau_j),\log(\sigma_j)\}$ are unknown
parameters.  Unlike \eqref{LM}, there is no intercept term in
\eqref{KF}; dropping the intercept term has been previously justified when using state
space models to analyze glucose sensor data
\citep[e.g.][]{dicker2013continuous}.  The parameters $\sigma_j,\tau_j$
control how heavily recent observations are weighted when estimating
$\alpha_j(t)$.

Similar to the analysis in Section \ref{sec:LM}, we fit \eqref{KF}
using (i) a combined model where $\Theta_j = \Theta$ for all $j$; (ii) an individual model where
$\Theta_1,\ldots,\Theta_p$ are estimated separately; and (iii) a
nonparametric mixture model,  where $\Theta_j
\sim G_0$ are iid draws from a $d=2$-dimensional mixing distribution.
Under (i)--(ii), $\sigma_j$ and $\tau_j$ are estimated by maximum likelihood and $\widehat{\mathrm{FS}}_j(t_i) =
\mathbb{E}\{\alpha_j(t_i) \mid \mathcal{F}_{t_i}\}\times \mathrm{ISIG}_j(t_i)$, where the conditional
expectation is computed with respect to the Gaussian law governed by
\eqref{KF}, with $\hat{\sigma}_j$ and $\hat{\tau}_j$ replacing $\sigma_j$ and
$\tau_j$ (i.e. we use the Kalman filter).  For the nonparametric
mixture (iii), $\widehat{\mathrm{FS}}_j(t_i) =
\mathbb{E}_{\hat{G}_{\Lambda}}\{\alpha_j(t_i) \mid \mathcal{F}_{t_i}\}\times \mathrm{ISIG}_j(t_i)$, where
the expectation is computed with respect to the model \eqref{KF} and the
estimated mixing
distribution $\hat{G}_{\Lambda}$.   Results are reported in Table \ref{tab:BG}.

\subsection{Comments on results}

From Table \ref{tab:BG}, it is evident that the NPMLE mixture approach outperforms the
individual and combined methods for both the linear model and the
Kalman filter/state space model.  The Kalman filter methods perform substantially better than the
linear model, highlighting the importance of time-varying parameters
(scientifically, this is justified because the sensitivity of the
glucose sensor is known to change over time).  Note that all of the relative MSE values in
Table \ref{tab:BG} are greater than 1, indicating that $\mathrm{CGM}$ still outperforms
all of the methods considered here.  Somewhat more ad hoc methods for
estimating blood glucose density that do outperform CGM are
described in \cite{dicker2013continuous}; these methods (and CGM)
leverage additional data available to the continuous monitoring
system, which is not described here for the sake of simplicity.  The
methods in \cite{dicker2013continuous} are somewhat  similar to the ``combined''
Kalman filtering method from Section \ref{sec:KF}, where $\Theta_j = \Theta$ for all $j$; it would
be interesting to see if the performance of these methods could be further
improved by using NPMLE ideas.

\section{Conclusion}
\label{sec:discussion}

We have proposed a flexible, practical approach to
fitting general multivariate mixing
distributions with NPMLEs and illustrated the effectiveness of this
approach through several real data examples. Theoretically, we proved
that the support set of the NPMLE is a subset of the convex hull of MLEs when the likelihood $f_0$ comes from a class of
elliptical unimodal distributions.  We believe that this approach may
be attractive for many problems where mixture models and
empirical Bayes methods are relevant, offering both effective
performance and computational simplicity.

\section*{Acknowledgments}
The authors are extremely grateful to the referee for their helpful and stimulating comments, leading to an improved exposition. The work of Lee H. Dicker is partially supported by NSF Grants DMS-1208785 and DMS-1454817. The work of Long Feng is supported by National Institute on Drug Abuse Grant R01 DA016750.

\bibliographystyle{elsarticle-harv}

redpajama_set_name: RedPajamaArXiv


text:
<?php

use Silex\WebTestCase;

class controllersTest extends WebTestCase
{
    public function testGetLDP()
    {
        $client = $this->createClient();
        $client->followRedirects(true);
        $crawler = $client->request('GET', '/');

        $this->assertTrue($client->getResponse()->isOk());
        $this->assertContains('HTML RDF representation', $crawler->filter('body')->text());
    }

    public function createApplication()
    {
        $app = require __DIR__.'/../src/app.php';
        require __DIR__.'/../config/dev.php';
        require __DIR__.'/../src/controllers.php';
        $app['session.test'] = true;

        return $this->app = $app;
    }
}

redpajama_set_name: RedPajamaGithub


The Washington Wizards head to BMO Harris Bradley Center to take on the Milwaukee Bucks on Tuesday, February 27, 2018. The opening line for this game has Milwaukee as 4 point favorites.
The Wizards come into this game with an overall record of 35-25. They are 1st in the Southeast Division. They currently have a point differential of 1.9 as they are averaging 107.5 points per game while giving up 105.6 per contest. The Bucks have an overall mark of 33-26. They currently are 3rd in the Central Division. They are averaging 105.2 points per contest and giving up 105.2, which is a point differential of 0.0.
Otto Porter averages 14.6 points per contest in 31.9 minutes per game this year. He averages 2.1 assists per game and 6.4 rebounds per contest for Washington. His field goal percentage is 50% for the season while his free throw percentage is 84%. Bradley Beal comes into this game averaging 23.7 points per contest while playing 36.5 minutes per night so far this year. He grabs 4.5 rebounds per contest and distributes 4.4 assists per game. His FG percentage is 46% while his free throw percentage is 80%.
Khris Middleton averages 20.1 points per game this year and he's on the court an average of 36.7 minutes per game. He also brings in 5.3 rebounds per game and 4.1 assists per contest. His field goal percentage is 46% and his free throw percentage is 87%. Giannis Antetokounmpo has scored 27.6 points per contest while playing 37.2 minutes per game this season for the Bucks. He hauls in 10.3 rebounds per contest and has 4.9 assists per game on the season. His FG percentage is 54% and his free throw percentage is at 76%.
The Wizards allow 105.6 points per contest which is 14th in the NBA. They also surrender a 46% shooting percentage and a 34% average from 3-point range, ranking them 11th and 3rd in defensive field goal percentage. Opponents average 43.1 rebounds a game which ranks 17th, and they allow 21.8 assists per game which puts them in 9th place in the league. They force 15.1 turnovers per game which ranks them 25th in the NBA. Washington averages 107.5 points per contest this year, which is 11th in the league. Their average scoring margin is 1.88 and their shooting percentage is 47% as a unit, which has them ranked 8th. From 3-point territory they shoot 37%, which is good for 8th in the NBA. They average 42.9 boards per contest and 24.6 assists per game, which ranks them 20th and 5th in those offensive categories. In the turnover department they are 15th with 14.3 per game.
The Bucks give up 105.2 points per game, which ranks them 11th in the league. Their defense is surrendering a 47% shooting percentage and a 36% average from behind the 3-point line, ranking them 19th and 14th in those defensive statistics. They rank 14th with 23 dimes allowed per game and 8th with 42.1 boards surrendered per contest. The Bucks are 27th in the league in forcing turnovers with 15.5 per contest this year. Milwaukee is 17th in the NBA with 105.2 points per contest this season. They are 5th in the league in shooting at 48%, while their average scoring margin is -0.02. They shoot 35% from beyond the arc, which is good for 24th in the league. They are 30th in the NBA in rebounding with 39.6 boards per contest and 19th in assists per game with 22.5. The Bucks are 22nd in turnovers per contest with 13.8.
The Wizards are 8th in the league in defensive rating with 108, and their offensive rating of 110 is ranked 18th. They are 15th in the NBA in pace with 97 possessions per 48 minutes, and their eFG% of 53% is 11th. As a unit they are 12th in the league in true shooting percentage with 56% and they shoot 3-pointers on 31% of their shot attempts, ranking 23rd in the NBA. Washington is 18th in turnovers, as they turn the ball over an average of 13 times per 100 possessions while forcing 13.7 on defense. Their defensive eFG% for the year currently is 52%, which is 21st in the NBA.
Milwaukee is 17th in the NBA in turnovers, as they turn the ball over an average of 13 times per 100 possessions while forcing 14.2 on the defensive end. Their defensive effective field goal percentage for the season currently is 52%, which is 15th in the league. The Bucks are 12th in the NBA in offensive rating with 109.1, and their defensive rating of 109.1 is ranked 14th. In terms of pace they are 24th in the league with 95.7 possessions per 48 minutes, and their effective field goal percentage of 53% is 8th. As a team they are 6th in the NBA in true shooting percentage with 57% and they shoot 3-pointers on 30% of their shots, ranking 25th in the league.


Decorating your cupcakes and tray bakes has never been so magical. Use this legendary creature to dust your sweet treats with sparkle and glitz for an instant dazzling effect. A mythical fantasy turned into an edible reality that no one will be able to resist!
Fun sprinkler for cake decorating. Fill and get decorating!


text:
Notley fed up with federal inaction on pipeline
January 15, 2019	Don Horne
Alberta Premier Rachel Notley expressed frustration again Monday about a lack of progress in completing the Trans Mountain pipeline.
Notley's comments came after a federal cabinet shuffle that left Natural Resources Minister Amarjeet Sohi in his portfolio.
She said that's probably a good thing as removing Sohi before the outcome of a court-ordered Indigenous consultation by the National Energy Board would be likely to cause even further delay.
"Albertans still need the federal government to step up and support the industry while we are trying to get through this ridiculousness of having not enough capacity to get our oil and gas to market," Notley told Canadian Press. "We've talked about that. We've talked about rail. We've talked about other interim programs that could come into place, and we're disappointed we've not heard anything from the federal government yet."
Notley announced late last year that her government will buy rail cars to transport an additional 120,000 barrels a day, which would increase the amount of oil being moved by rail in Canada by one-third.
Discussions about a purchase agreement are ongoing, she said.
She was asked whether the current Liberal government should receive another mandate from voters in a federal election expected later this year.
"The facts of the matter is that the previous government, which happened to be from a different political party, also didn't get it done," she said. "Quite honestly, the considerations that have led to the ridiculousness that Albertans are so frustrated with right now … has been in the making for decades."
The federal government bought Trans Mountain and its expansion project for $4.5 billion last summer only to have the Federal Court of Appeal strike down the energy board's approval. The court said there had been inadequate Indigenous consultation and failure to consider impacts on the marine environment.
The board's final report needs to be submitted to the federal cabinet by Feb. 22.
Notley said partisanship should have nothing to do with support for the resource sector, which is an economic engine for the rest of Canada.
The current federal government has at least accomplished one thing, she said.
"There was decades of factors that were at play which led to the instability that jeopardized that project. We did get the federal government to buy the pipeline. That's not nothing. There is a vested interest they have now in getting this darn thing built."
(Canadian Press)
ABB Completes Acquisition of K-TEK
Schneider Electric Announces New Canadian President
Technology Profile: Buzz about medium voltage
Keystone XL pipeline to be "sold as scrap" if cancelled
Growing presence in Canadian power generation has new director
RCMP reviewing procedures after pipeline blockade arrests

redpajama_set_name: RedPajamaCommonCrawl


Brian Tracy first came on my radar screen many years ago when I purchased his cassette series called the Psychology of Achievement. Looking through my archive of training materials I still have it to this day. Just a few weeks ago, I was honored to be able to share three days with Brian at the 20 year anniversary of the Success Mastery Academy put on by good friend Kyle Wilson.
Over the three days, Brian taught many new concepts; along with bringing back some of the old stalwarts and principles that needed to be pulled back top of mind. There were so many great reminders that I thought that I'd share a few with you here.
Brian is a huge proponent of constant and never-ending personal improvement. That we all have untapped and unfulfilled possibilities. The point that resonated very early was that all the secrets on how to be successful have already been found. And then if you're not acting on those secrets are simply not taking the time to learn them…because they are no longer secret!
He suggested you consistently ask yourself the question, when you're looking at a high achiever, "What are they doing differently than I am?" That makes a tremendous amount of sense, he also said, "Your mind changes when your mind collides with a new idea." It means you have to constantly be questioning what you're doing, how you're doing it, and if there may be an alternative way.
Another concept he raised was that 100% of people work on commission. His justification of that statement was that we all receive a percentage of our results. Some officially because they're paid on a percentage commission rate, while others are paid on hourly rate that represents their contribution towards results. I know I had never looked at it that way prior to hearing him say it. Again, makes total sense!
When talking about high achievers, Brian stated, everything counts…everything you do is either raising or lowering your self-esteem. Also the more often we think the thoughts with an emotional charge, they will ultimately manifest.
One final concept to touch on here is three simple questions that allow you to truly hear someone's inner self… in other words, who do they really think they are…they will reveal themselves if you let them. Question #1 ask: How do you see yourself? Their answer…I am _____, _____ and _____. Question #2 ask: How do you see other people? Their answer…People are _____, _____ and _____. Question #3 ask: How do you see life? Their answer…Life is _____, _____ and _____.
When asked, "How do you see yourself?" if they answer, "I am intelligent, funny and outgoing gives you a totally different impression than if they say, "I am average, blah, and introverted.
When asked, "How do you see other people?" if they answer, "People are interesting, fun and great to be with, gives you a totally different impression than if they say, "People are boring, dull, and needy.
Take a true interest in those you come across and do your best to add value to their lives every time…in turn it will add value to your life!


Europe and the Mediterranean brim with some of the world's most iconic cruise destinations; captivating travellers with the perfect balance of history, culture and natural beauty. Our Finest Collection is the perfect way to experience it for yourself - gaining authentic insights during kayaking tours, dips in geothermal pools and cruises between unforgettable ports.
Europe and the Mediterranean offer some of the most diverse and culturally-rich experiences anywhere on Earth, and our Finest Collection ensures you won't miss any of them. Experience Northern Europe's astonishing natural landscape, captivating history and rich maritime tradition on a fjords cruise. In the Mediterranean, explore volcanic landscapes in Lanzarote, immerse yourself in Athens' ancient Greek history and watch the sun set over bright white cliffside tavernas in Santorini. And this is all alongside extended hotel stays and unique excursions.
Our Finest Collection packages are the perfect way to experience the very best of these diverse cruise destinations, combining seaborne discovery with unforgettable land adventures. Whether you are craving a Croatian odyssey or an island-hopping adventure in Greece, our tailor-made packages will turn your travel daydreams into reality.


text:
Trion to launch fund with $100M in buying power
Company is moving from real estate syndication to private equity model
TRD LOS ANGELES /
Mar.March 14, 2016 11:30 AM
Trion Properties, the Los Angeles-based multi-family investment firm, is moving into the real estate fund business.
The company, headed by Max Sharkansky and Mitch Paskover, plans to raise a $30 million equity fund with $100 million in buying power to invest in multi-family assets in L.A., the Bay Area, San Diego and Portland, CoStar reported.
That means that, rather than raising money through syndication one deal at a time, the firm will be asking investors to trust that it will find them the right deals. It also gives Trion the ability to pull the trigger on a transaction at a moment's notice.
"Our firm has a deep pipeline of opportunities that we've already identified, and the shift to a private equity fund will allow us to take full advantage of these opportunities as they arise, increasing activity in our target markets," Sharkansky told CoStar.
Trion, whose current portfolio includes 720 units across those four aforementioned markets, says its edge is in its narrow focus on undermanaged multi-family properties.
"Because we're so focused, we bring a deeper expertise in this product type and geography than our competitors," Paskover said. "By targeting urban infill projects, we're limiting the downside and ensuring that our assets can weather economic storms."
The company recently nabbed the Avalon, a 47-unit building in the heart of Koreatown, for $7.54 million, or $329 per square foot. The seller was Vista Investment Group from Santa Monica. [CoStar] – Katherine Clarke
Start your timer: Leonard Ross has 4 months to sell Hearst mansion
The youth are fueling Sam Zell's outdoor home rental business
Metro Networks founder Saperstein lists Malibu home for $115M
Is third time a charm for state's high-density housing bill?
$1K a night, easy: Miami area hotels score high rates for Super Bowl LIV
7083 Hollywood Blvd, 2nd Floor, Los Angeles, CA 90028


This is a family owned business. Our welcome to you is sincere and limitless. We were born and raised here, we love and know this land, its people and traditions. We are truly local and we will do our best to make you feel local too. You stay with us – you are our family.


I've been obsessed with pretty stationery for as long as I can remember. I remember the day when my mom let me go into town for the first time ever with my cousin, without any adults. I was 11 years old, and what did I come home with? Some rubbers and pencil sharpeners. This happened every week, but I began to get a little adventurous as the weeks passed and mixed it up a little, throwing a few pens and notebooks in there for good measure!
This love has always stuck with me, and now, 16 years later, although I don't buy rubbers anymore, I'm still a sucker for notebooks and pens and pretty filing boxes to stick all our bills in!
When leaving London last week, we had a bit of time to kill whilst waiting for our train, so I wandered into PaperChase at Kings Cross and swooned over all their pretty items. I could have bought so much but I do like to be careful with my money. However I was feeling a little flush so had a small splurge and treated myself to this gorgeous recipe book and a pack of pretty floral pens. These pens are actually gel pens and are the same colours as the big flower on the end of each pen. This may be a no-brainer to you, but I assumed they were all biro style standard blue or black pens. I probably wouldn't have purchased them otherwise because I'm not a fan of gel pens (although the ink flows quite nicely) (did I actually just speak about the flow of ink then?). I'm sure I'll have fun writing in pink though!
The recipe book is divided into lots of different sections and has a few handy plastic wallets at the back to pop any recipes you may rip out of magazines (which I'm forever doing). It also has a zip-lock bag which again I'm sure will come in handy for notes.
I've always wanted one of these but never got round to getting one... only thing left to do now is decide what recipes to put in them. You can only fit 10 recipes per section in, so I'm going to have to think wisely on which recipes will make the cut!
Have you got any recipe recommendations for me?! There are so many out there that I find it all a bit too overwhelming to the point of not knowing which one to choose, therefore I wont choose any!


text:
World of Warcraft Retail
What To Do At Level 70? Dragonflight Endgame Explained
Level 70 Endgame Dragonflight WoW
January 04, 2023 12 minutes
Guide Navigation
Finish The Main Storyline
Start Making Friends: Reputation Leveling
Fully Train Your Dragon
Hunt for Treasure and Rares and Events
Delve into Dungeons and Raiding: PvE
Destroy everyone and everything: PvP
Get Comfy and Just Craft: Professions
Start Over: Leveling New Character
You finally dinged level 70 in Dragonflight. You've reached the end game, my friend, sitting there lost as a gnome in a big and tall store. What exactly is there to do at level 70 in Dragonflight? Where do you begin? In this guide, I aim to answer all these questions by going over the several approaches to World of Warcraft you can take upon hitting the max level in Dragonflight. So let's soar on into the Dragon Isles, and let's get started because we got a lot to cover with this guide, and hopefully, by the end, you'll be less confused.
Upon hitting level 70, you'll likely have gotten to level cap without actually finishing the main storyline on your first character, as well as most of the side quests in Dragonflight scattered across the Dragon Isles. I know that you don't give a damn about the story or the lore and all that nonsense, but I got news for you: you have to start giving a damn because the first thing you need to do upon hitting max level is finished at least the main questline. Yes, I know you don't care about the lore, you just want to raid and crack skulls, but you got to finish the main questline. Completing the main quest line in Dragonflight will allow you to progress the story of Dragonflight further for those of you who care about that. However, you must also go into the Valdrakken storyline to unlock relevant end-game features like the Spark of Ingenuity. But more on that later. Finishing the main storyline and side quests is also incredibly important for gaining a reputation with the various factions throughout the Dragon Isles, like the Tuskarr and the Centaur, and this leads us to what you should do after you finish at least the main storyline.
Four major factions have been introduced to us in Dragonflight. The first is the Dragonscale Expedition. These are a bunch of explorers of the Dragon Isles. There's the Tuskarrs, aka the best faction in this expansion. They just want to catch fish, eat soup, and then sleep off their blubber in this cozy little house on the island's south end. There's the Centaur - a bunch of smelly Centaur Clans who just want to hunt big game and kick big ass. Finally, you have the Valdrakken Accord.
What you need to be focusing on as soon as you get to level 70 is gaining a reputation with all these guys, earning you a ton of benefits and cool little rewards across the Dragon Isles: pets, mounts, gear, and even just fine new gameplay mechanics like having a hunt companion from the Centaur or even the ability to rock climb from the Expedition. One of the best ways early on to get more reputation with the factions is to do all of the side quests I mentioned earlier. The quests are not a part of the main storyline of Dragonflight. And since you're level 70, you're probably gonna tear through them ultra-fast. It's also a great way to get the Loremaster achievement, and who doesn't want that? You can also get a lot of reputation with the various factions by doing the world quests scattered throughout the Dragon Isles. The best part is that if you're just now hitting level 70, the world quests reward some decent gear for getting you started. It's also a great way to get some quick gold and Primal Chaos for crafting. But we'll talk about crafting later. Progressing in the reputation will also unlock more World Quests, potential gear rewards, and other rewards. Tired of leveling the reputation yourself? We provide a reputation boost service. It all will be done in a few days when you are asleep or at work, so you will not even notice anything but the progress at the end. But if you're gonna get these factions to max level fast, you're gonna need a fast-as-hell mount, leading us to number three.
It's likely that many of you guys, even before 70 took to the skies and completely pimped out your dragon. But you may hit Level 70 and still have many Dragon Glyphs to find around the open world. If that's the case, go out and collect all of the glyphs while enjoying the beauty of the Dragon Isles and what it offers.
Leveling up your dragon will also earn you more Vigor points and aerial abilities, making getting around the Dragon Isles much faster and easier. This is a must. Getting around the world quickly is so damn important, especially if you're a treasure hunter like me and speaking of treasures that lead us to number four.
Remember the Dragonscale Expedition I mentioned earlier? These guys have secretly stashed all of their findings in these little explorer packs you can find throughout the Dragon Isles. There is also other stuff in the packs, like Primal Chaos, Dragon Isles Supplies, and other valuable things for you to enjoy. So be on the lookout for explorer packs as you look for treasure throughout the Dragon Isles. Also, keep your eyes open for rares while flying above the Isles. Rare spawns will occasionally appear across the Isles, dropping epic chests for you to dig into to see what's inside. Typically a piece of epic gear if you're lucky. You can even find the World Boss event, which you can participate in with big groups here and there. Look for the little icons on your mini-map as you fly around. There are also significant community world events to take part in too that you'll find, like the Siege of Dragonbane Keep, which can earn you some cool gear. The Tuskarrs soup event south of the Azure Span will get you a significant buff. As well as Primal Storms all of which offer rewards gear and reputation. The Primal Storm gear can also be upgraded to item level 385, an excellent way to gear up for raiding fast. Now let's move on to the group content.
Dungeons are such a fun part of World of Warcraft. They're a part of its backbone, and Dragonflight delivers some excellent dungeon content experiences. Especially watching people get splashed back by the waves near the end of the Halls of Infusion dungeon. Upon hitting level 70, after gearing up a little from some world quests or trolling the auction house, get yourself into some heroic dungeon content using the group finder. When you can jump into some mythic dungeons to really push your item level.
As season one has started to dive into mythic plus dungeons as well. Some older dungeons are now available for us to re-explore, like the Shadowmoon Burial Grounds and the Halls of Valor from Legion. Once you find some friends and a friendly guild, do yourself a favor and check out the Vault of the Incarnates raid. In this raid, you will be putting an end to that mutant angry duck dragon that accidentally put a fork into an electrical outlet that's been plaguing us since the beginning of the expansion. This is all about finding friends in the game. I know that it's easier to say than to do sometimes. Thus, we are delivering great mythic+ and raid services. Arguably we are one of the top service providers on the market. You can check out our raid services and our mythic plus services if you are interested. I know some of you guys don't care about raiding, you don't care about dungeons, but you do care about killing things, and that leads us to number six.
Did you just get done playing through God of War Ragnarok, and you still want to rip off some heads like Kratos tearing off the head of Apollo? Well then, you're in luck because Dragonflight delivers.
If you're someone like me, you're probably a big fan of open-world PvP - screw this whole Alliance and Horde working together nonsense - this is World of Warcraft, not World of Get Along and Make Crafts. Luckily for us in Dragonflight, we are blessed with a bloody token system that allows us to earn gear and rewards from mindlessly slaughtering people out in the open world just as God intended. The open-world PvP war mode system must be addressed in WoW thus far and needs more updates. Where are the perks of sieging cities and towns? Flying around and ganking the occasional player is excellent, but what about the joy of giant armies fighting over territory? If world PvP is not your thing, though, do not worry. We've got Battlegrounds, Rated BGs with your guild mates, Arenas, and even Solo Shuffle. We even got a new arena with this expansion. If you have tried some PvP and want to bring your PvP skills to the next level, we offer a PvP coaching service where you will be playing alongside Gladiators and R1 players. So if you just want to crack some Alliance skulls or collect some Horde femurs, dive on in and have fun. But if you just want to get comfy - I understand, leading us to number seven.
Maybe throughout this guide, you've just been thinking, I don't want to do dungeons, I don't want to raid, I don't want to crack skulls. I don't want to do any of this crap. I just want to chill. I just want to pick flowers and craft myself some awesome-looking gear. Well, my friend, if you're like me and just love crafting - go for it. Dragonflight is an expansion that has wholly overhauled crafting to keep you engaged for months to come. You can do anything from potions meant to put more fun into the game to relevant and craftable epic giant shoulder pads.
As you travel through the Dragon Isles upgrading your dragon or just gaining that reputation, you will find crafting recipes scattered throughout the world, allowing you to make all sorts of exciting and fun items for your character or your friends. Really depending on who you ask, professions are in a really excellent place right now in World of Warcraft. The only con is that it's a shame that crafting orders are not being used more. Blizzard's going to have to work on that system. But that's a topic for another time. Make sure if you go down the crafting route to keep up to date with the Spark of Ingenuity. If you need to learn what this is - it's a system in the game where every so often, you get a Spark of Ingenuity that allows you to craft really high-level gear. Just make sure you stay up to date with this. Have fun with professions. You can do it if you want to be a crafter who gears up purely through crafting. And if all of that doesn't work for you, if you just hate everything I've talked about on this list and want something more refreshing, new, but still familiar, I got one last bullet for you.
Only some people care about the end game. Some of us want to keep exploring the World of Warcraft, the quests, and the locations and discover things we may have missed the first time going through the game. If you're not a fan of end-game and have yet to care about anything I've described with this guide, why not start over, make an alt, and level to max again.
Dragonflight is the perfect expansion for leveling new alts. In this expansion, there are way too many quests meaning that you can level a new character and only do some of the quests on the Dragon Isles. Once you finish the main story on your primary character, you can explore this new continent in any zone order you want. You don't even need to do the main story again once you've done it on your main. Also, Dragonriding progression carries over to your other characters, so don't worry about finding all those damn glyphs again.
As you can see, there's a lot to do with the end game in Dragonflight. Whether you're a PvPer or a Raider, we will all be satisfied. If you're planning on leveling a new character but don't quite enjoy the process or simply don't have enough time for it, check out our fast and cheap leveling service. Thank you all for reading, and I will see you all soon in the Dragon Isles!


text:
Steam for Linux now open to all users
Valve has announced that Steam for Linux is now open to all users.
The Steam for Linux beta client is now available to all Steam users, so if you've been patiently waiting for an invitation to join us, consider yourself officially invited!
With a growing catalog of Linux-supported games, an active Steam for Linux community group, and a new GitHub bug reporting repository, the timing's right to jump in and share your feedback.
If you've already downloaded and installed Steam for Linux beta, you should redownload and install the latest deb available on its website, because Steam now has its own Ubuntu repository that's going to be used for updates. Well, at least in theory, because as soon as I've installed the new deb and launched Steam, it started to download an update through the client.
Supported games include Serious Sam 3: BFE, Team Fortress 2, Killing Floor and others. For a list of games supported by Steam for Linux, see THIS page.
Download Steam for Linux (deb)
[via WebUpd8]
Time Posted: 5:57 PM Labels: Games, Linux
YouTube's Top Ten Viral Videos of 2012
Pumpkin's Palette: Yamaken
2012 Top 10's of the Anime World
Facebook plans autoplay video ads
Introducing the Humble Indie Bundle 7
Download your complete Twitter archive


JERUSALEM, Nov. 4 (Xinhua) -- Israel's Energy Ministry announced on Sunday that it will launch a tender for gas and oil exploration in the Mediterranean Sea, with the aim of exporting most of the energy found, in view of the low demand in domestic market.
This is the ministry's second attempt to find such investors, about a year after the failure of the first round.
In 2016, the ministry announced the opening of the sea, off the coast of Israel, to gas and oil exploration, after four years in which the sea was closed to competition.
The closing took place after companies not involved in the Israeli energy market took advantage of the licensing policy to search areas with an unclear purpose.
On the other hand, the closing fixed the status of the "Tamar" and "Leviathan" gas drill fields in the Mediterranean Sea as a duopoly in the gas supply market.
The ministry, which reopened the sea for searches, hoped that with the regulation of investors and approval of a government gas pipeline, investors would rush to Israel, just as they rush to tenders in Cyprus, Egypt and Lebanon. In practice, no such investors came.
The announcement of the results of the first tender was rejected three times due to a small number of participants. Finally, the Greek company Energean Oil & Gas and a consortium of four Indian government companies agreed to obtain search licenses.
The Greek company plans to start the search soon, while the Indian companies have not yet done anything about it since the license was granted.
Despite the failure of the first tender, Israeli Minister of Energy, Yuval Steinitz, decided to launch another one so as not to lag behind tenders in neighboring countries and the dream of turning Israel into a regional supplier.
In addition, according to existing studies in the economic waters of Israel, there is another 2 trillion cubic meters of gas, and also a probability of finding oil.


We provide a world class car rental service in Kolkata, India.
We focus on delivering great quality and value for money!
Our car service is the result of our dream to build a company which achieves customer loyalty through understanding ever-changing needs, delivering quality, and providing value for money.
We focus on human resource development, transparency, and the involvement of team members in decisions, resulting in a team of highly creative, knowledgeable team members with impeccable integrity.
Through extensive use of technology, we keep our operating costs low and pass on the savings to our customers with competitive prices.


text:
Hetti Barkworth-Nanton, Chair of the Board at Refuge, on tackling domestic abuse
>>> Click here to play this episode <<<
Chair of the Board at Refuge, Hetti Barkworth-Nanton, on tackling domestic abuse. 1 in 4 women in England and Wales will experience domestic abuse in their lifetime.
A wide-ranging conversation on the topic of domestic abuse. Hetti sheds light on sobering statistics, attitudes and behaviours and describes the challenges presented by COVID lockdowns and how new technologies are impacting the field.
In England and Wales, around 2 women a week are killed by a current or former partner, and 1 in 4 women will experience domestic abuse at some point in their lifetime.
The problem goes beyond domestic violence. As Hetti points out, coercive control is the dark side of domestic abuse and often leads to the worst outcomes.
Refuge is an organisation helping those who experience domestic abuse. They run the National Domestic Abuse Helpline and also have deployed a Tech Abuse Team, who support women who are experiencing online abuse and other tech-related abuse.
Innovations in technology present serious challenges – innovations leading to the miniaturisation of surveillance technology, for instance, or software that can compromise WiFi connectivity and remotely lock doors and override temperature control and other features in one's own home.
This episode explores domestic abuse from various angles and provides invaluable information on the state of affairs and opportunities for improvement.
Thank you for downloading this episode of The Do One Better Podcast. Please leave us a rating and a review to help others find this show.
About Hetti Barkworth-Nanton
Appointed to the board in 2020, Hetti became Chair of Refuge's Board of Trustees on 1st June 2020.
Hetti has a career spanning British Airways, Centrica, BT and Vodafone, leading financial, commercial and transformational change, often enabled by leading edge technology.
She became the CEO of Ploughshare Innovations in April 2020 and is renowned for leading cultural and commercial transformation, and her highly creative, collaborative and engaging approach with teams and partners. She is also a recognised champion for diversity, equality and inclusion.
Hetti has been an active campaigner raising awareness about domestic abuse and the criminal justice system since her best friend, Joanna Simpson, was killed by her estranged husband. Hetti is the founder and former Chair of the Joanna Simpson Foundation and was previously a Pioneer at Safe Lives.
As CEO of an organisation which is also on a high growth trajectory, and Chair of Refuge which is going through enormous change whilst seeing ever increasing demand for services, Hetti focuses her spare time enjoying long walks in the countryside with her family and the dogs.


The East Austin Studio Tour is a free, annual, self guided art event spanning two weekends in November. EAST provides opportunities for the public to meet the local artists and artisans of Austin in their creative spaces.
New to the tour, the EAST app is a digital companion to the catalog and map powered by Third Rail Creative. The app has lots of great features to help you get the most out of your tour including favoriting features, interactive map, calendar of events and more. Search East Austin Studio Tour in the Apple Store and Google Play.
Free maps will be available at all 23 branches of the Austin Public Library starting Oct 11.


Here you can find important and useful information for your preparations for Secutech Vietnam.
The exhibition is a great platform to introduce and allow people to see what we are showcasing. The fair has a well-balanced number of large to small and medium size exhibitors/companies. The number of visitors we were able to meet went far beyond our expectations. The customer service was also well organized and clear which makes setting up for the show much easier.
The 2019 exhibitor manual will be released 3 months prior to the show.
This manual includes rules, regulations, technical details and other important information for designing, settling-up spaces and shipment to be followed.
The Raw Space option does not include any stand design, fittings and carpeting, but enables the exhibitors to create their own individual stand build.
Shell scheme is a basic stand framework build by our appointed stand contractor. For companies with a limited amount of time and budget.
* Shell scheme is a basic stand framework build by our appointed stand contractor. For companies with a limited amount of time and budget.
You would like to exhibit at Secutech Vietnam 2019? Register your stand now!
Download the brochure to learn about Secutech Vietnam 2019.
Tentative schedule for set-up and dismantling is as below, subject to change.
1. Air-conditioned equipment is supplied during the show days (14 – 16 August 2019).
2. Visitor registration counter will be closed 30 mins before the show closing hours.
3. All exhibitors are strictly prohibited to move-out before 17:00 on 16 August 2019.
4. The organiser will provide the construction schedule for each exhibitor on move-in day (TBA).
Please be sure to follow the regulations.
5. The organiser reserves the right to renew the schedule and contents above.
Please take notice of the letters from the organiser and keep updated with Secutech website.
Secutech Vietnam provides comprehensive marketing campaign that gives you year-round global exposure to your target markets.
The show directory is an effective tool to maximize your exposure and extend promotion results in Secutech Vietnam. It is the official publication by the organiser which covers comprehensive exhibitor's company information. This bilingual show guide comes handy to international and local visitors for searching and better understanding the showcases on the floor. 10,000 copies of the show directory will be distributed during Secutech Vietnam, as well as to major distributors, government departments and end users in Vietnam.
The Secutech Vietnam team is happy to answer any question or special request. Please drop us an email or call us directly in any matter.


So, your shopping cart processes some or all of its payments from PayPal. Now you want to get all that transactional data into Quickbooks.
If you've done some research, you know that it doesn't work very well to export the PayPal transactions using their export feature, then import the IIF file into Quickbooks. Some data will not get transferred into Quickbooks, and some data will get put into the wrong fields.
You might also be struggling with an inadequately programmed export-to-Quickbooks feature in your cart. For integrating your cart and PayPal with QB, here are a few vendors that created software which may help you.
2. BigRed Consulting. Not nearly as refined as Simpleport.
Those two are purely focused on the PayPal to QB conversion. The next two are larger software packages that do the PayPal export to Quickbooks import, and much more. They fully integrate a multitude of shopping carts (including Magento, X-Cart, Zencart, Prestashop, osCommerce, and more) with Quickbooks.
3. ECC by Webgility. Not that expensive for what you get. Highly refined. Excellent support according to cart forum commentary by actual users.
4. T-Hub by Atandra. More expensive, but a review of comments on cart owner forums is spotty. Some comments are positive and some are negative on issues ranging from support to bugs.
This blog post will be updated continually. Please share it and bookmark it to be notified of updates.
Hi! I've been reading your website and giving you a shout out from Porter Texas! Just wanted to mention keep up the great work!