text:
\section{Introduction}

In some finite mixture distributions maximum likelihood estimator (MLE)
does not exist.  Let us consider the following example.  Denote a
normal mixture distribution with $M$ components and parameter
$\theta=(\alpha_1, \mu_{1}, \sigma_{1}^{2} ,\ldots,\alpha_{M},
\mu_{M}, \sigma_{M}^{2})$ by
\begin{eqnarray}
 f(x;\theta)
  =
  \sum_{m=1}^{M}
  \alpha_{m} \phi_{m}(x; \mu_{m}, \sigma_{m}^{2}) ,
 \nonumber
\end{eqnarray}
where $\alpha_{m} \; (m=1,\ldots,M)$
are nonnegative real numbers that sum to one and
$\phi_{m}(x; \mu_{m}, \sigma_{m}^2)$
are normal densities.
Let $x_{1},\ldots,x_{n}$ denote
a random sample of size $n \geq 2$
from the density $f(x;\theta_{0})$.
In view of the identifiability problem of mixture distributions
discussed below, here $\theta_{0}$ is a parameter value designating the
true distribution.  However for simplicity we just say
$\theta_{0}$ is the true parameter from now on.
The log likelihood function is
\begin{eqnarray}
 \sum_{i=1}^{n}
  \log{f(x_{i};\theta)}
  =
  \sum_{i=1}^{n}
  \log{
  \left\{
   \sum_{m=1}^{M}
   \alpha_{m} \phi_{m}(x_{i}; \mu_{m}, \sigma_{m}^{2})
  \right\}
  }.
 \nonumber
\end{eqnarray}
If we set $\mu_{1} = x_{1}$,
then the likelihood tends to infinity as
$\sigma_{1}^{2} \rightarrow 0$.
Thus MLE
does not exist.

But when we restrict
$\sigma_{m} \geq c \;(m=1,\ldots,M)$
by some positive real constant $c$,
we can avoid the divergence of the likelihood.
Furthermore,
in this situation,
it can be shown that MLE
is strongly consistent
if the true parameter $\theta_0$
is in the restricted parameter space.

On the other hand,
the smaller $\sigma_{1}^{2}$ is,
the less contribution $\phi_{1}(x; \mu_{1}=x_1, \sigma_{1}^{2})$
makes to
the likelihood at other observations $x_{2},\ldots,x_{n}$.
Therefore an interesting question here is whether we
can decrease the bound $c=c_n$ to zero with the sample size $n$ and
yet guarantee the strong consistency of MLE.
If this is possible, the further question is how fast
$c_n$ can decrease to zero.

This question is similar to the (so far open)
problem stated in \citet{H1985},
which treats mixtures of normal distributions
with constraints imposed on the ratios of variances
while our restriction is imposed on variances themselves.
See also a discussion in section 3.8.1 of \citet{M2000}.

In the above example, the normality of the component distributions is
not essential and the same difficulty exists for finite mixtures of
general location-scale distributions such as mixtures of uniform
distributions. Furthermore in this paper we allow that each component
belongs to different location-scale families.  Let $\sigma_{m}$
($m=1,\ldots,M$) denote the scale parameters of the component
distributions and consider the restriction $\sigma_{m} \geq c_{n} \;
(m = 1, \ldots, M)$.  Then a question of interest here is whether we
can decrease the bound $c_n$ to zero.

For the case of mixture of uniform distributions, in \citet{TT2003-20}
we proved that MLE
is strongly consistent if $c_{n} = \exp(-n^d)$, $0< d<1$.  Here $d$
can be arbitrarily close to 1 but fixed.
In this paper, we prove that the same result holds for general finite
mixtures
of location-scale distributions under very mild regularity conditions
(assumptions  \ref{assumption:1}--\ref{assumption:4} below).
We employ the same line of proof as in
\citet{TT2003-20},
but the proof for the general finite
mixture is much more difficult.
As discussed in section
\ref{sec:discussion} the normal density satisfies the
regularity conditions and our result implies that
MLE is strongly
consistent for the finite normal mixture
if $\sigma_m \ge c_n = \exp(-n^d)$, $0<d<1$, $m=1,\ldots,M$.

Our framework is closely related to the method of sieve
(\citet{G1981}).
In the sieve method
an objective function  is maximized
over a constrained subspace of parameter space
and then this subspace is expanded to the whole parameter space
as the sample size increases.
Some applications and
consistency results for the method
are given in \citet{G1982}.
MLE based on a sieve is called a sieve MLE.
The convergence rates of sieve MLE
for Gaussian mixture problems are studied in
\citet{GW2000} and \citet{GV2001}
and their ideas are very interesting.
They obtain the convergence rates by
bounding the Hellinger bracketing entropy of
subsets of the function space
and assume that the corresponding subsets of the parameter space are compact
so that their bracketing entropy does not diverge.
In the case of sieve MLE, the approximating subspaces are usually
taken to be compact, whereas
we treat a sequence of non-compact subsets of the parameter space
expanding  to the whole parameter space as the sample size increases.
Therefore results on sieve MLE are not directly applicable in
our framework.

The organization of the paper is as follows.
In section
\ref{sec:preliminaries} we summarize some preliminary descriptions.
In section \ref{sec:main} we state our main results in theorems
\ref{kappa-lambda:thm} and \ref{main-thm:thm}.
Section \ref{sec:proof}
is devoted to the proof of theorems and lemmas.
Finally in section \ref{sec:discussion} we
give some discussions.

\section{Preliminaries on strong consistency and identifiability of mixture distributions}
\label{sec:preliminaries}

A mixture of $M$ densities with parameter
$\theta=(\alpha_1, \mu_{1}, \sigma_{1},\ldots,\alpha_{M}, \mu_{M}, \sigma_{M})$
is defined by
\begin{eqnarray}
 f(x;\theta) \equiv \sum_{m=1}^{M} \alpha_{m} f_{m}(x;\mu_{m}, \sigma_{m}) ,
  \nonumber
\end{eqnarray}
where $\alpha_{m}$, $m=1,\ldots,M$, called the mixing weights,
are nonnegative real numbers that sum to one
and $f_{m}(x;\mu_{m}, \sigma_{m})$, called the components of the mixture,
are density functions.  In this paper we consider the case that
the component densities are location-scale densities
with the location parameter $\mu_{m}\in \mathbb{R}$
and the scale parameter $\sigma_{m}>0$, i.e.
\begin{equation}
\label{eq:location-scale}
f_{m}(x;\mu_{m}, \sigma_{m})=\frac{1}{\sigma_m} f_m \left( \frac{x-\mu_m}{\sigma_m}; 0,1\right).
\end{equation}
As mentioned above, we allow $f_{m}(x;\mu_{m}, \sigma_{m})$ to
belong to different families.
For example, $f_{1}(x;\mu_{1}, \sigma_{1})$ may be  a normal density,
$f_{2}(x;\mu_{2}, \sigma_{2})$ may be a uniform density, etc.
Let $\Omega_{m}=\mathbb{R}\times (0,\infty)$ denote
the parameter space of the  $m$-th component $(\mu_m, \sigma_m)$
and let $\Theta$ denote the entire parameter space:
\begin{eqnarray}
 \Theta  & \equiv &
 \{
 (\alpha_{1}, \ldots , \alpha_{M}) \in \mathbb{R}^{M}
 \mid
 \sum_{m=1}^{M} \alpha_{m} = 1
 \; , \;
 \alpha_{m} \geq 0
 \}\times \prod_{m=1}^{M}\Omega_{m}.
 \nonumber
\end{eqnarray}

Let $\mathscr{K}$ be a subset of $\{1,2,\ldots,M\}$ and
let $\abslr{\mathscr{K}}$ denote the number of elements in
$\mathscr{K}$.
Denote by $\theta_{\mathscr{K}}$ a subvector of $\theta \in \Theta$
consisting of the components in $\mathscr{K}$.
Then the parameter space of subprobability measures
consisting of the components in $\mathscr{K}$
is
\begin{equation}
 \bar{\Theta}_{\mathscr{K}} \equiv
  \{
  \theta_{\mathscr{K}} \mid \theta \in \Theta, \sum_{m \in \mathscr{K}} \alpha_m
  \le 1
  \}.
  \label{eq:def:barTheta_scrk}
\end{equation}
Corresponding density and the set of subprobability densities are
denoted by
\begin{align}
 f_{\mathscr{K}}(x;\theta_{\mathscr{K}})
  &\equiv \sum_{k \in \mathscr{K}} \alpha_{k} f_k(x;{\mu}_k, \sigma_{k}) ,
  \label{eq:def:f_scrk} \\
 \mathscr{G}_{\mathscr{K}}
  & \equiv
  \{
  f_{\mathscr{K}}(x;\theta_{\mathscr{K}})
  \mid
  \theta_{\mathscr{K}} \in \bar{\Theta}_{\mathscr{K}}
  \} .
  \label{eq:def:scrg_scrk}
\end{align}
Furthermore denote
the set of subprobability densities with no more than $K$
components by
\begin{equation}
\mathscr{G}_{K} \equiv \bigcup_{\abslr{\mathscr{K}}\leq K}\mathscr{G}_{\mathscr{K}}
\qquad (1 \leq K \leq M).
  \label{eq:def:scrg_K}
\end{equation}

We now briefly discuss identifiability of parameters.
In mixture models, different parameters may designate the same
distribution.  When the component densities belong to a common
location-scale family, we can permute the labels of the components
and the distribution remains the same.  A mixture model of $K-1$
components can be obtained by setting one weight $\alpha_m = 0$ (with
arbitrary $\mu_m$ and $\sigma_m$) in a
model with  $K$ components.  These are trivial cases of unidentifiability
of parameters.
However there are more
complicated cases.  Let $U(x;a,b)$ denote the uniform density on the
interval $[a,b]$. Then, for example,
$\frac{1}{3}U(x;-1,1) + \frac{2}{3}U(x;-2, 2)$
and
$\frac{1}{2}U(x;-2,1) + \frac{1}{2} U(x;-1,2)$
represent the same distribution (\cite{EH1981}).
In this case the limiting behavior of MLE is not obvious,
although the estimated density should be consistent.
Therefore we first give a definition of consistency in terms of
the estimated density.

Let $f_0(x) = f(x;\theta_0)$ denote the true density and let
$\hat f_n(x)=f(x;\hat\theta_n)$ denote the estimated density.
\begin{definition}
\label{definition:1}
An estimator $\hat{f}_{n}$
is strongly consistent if
\begin{equation}
 \mathrm{Prob} \left(
  \lim_{n \rightarrow \infty}
  \normlr{\hat{f}_{n}-f_{0}} = 0
 \right) = 1 \; ,
\label{strongconsistency:def}
\nonumber
\end{equation}
where
$\norm{\cdot}$ is the $L_{1}$-norm.
\end{definition}

Although definition \ref{definition:1} is conceptually simple, in
order to prove the strong consistency of MLE
we work with the location and the scale parameters in
(\ref{eq:location-scale}) and the mixing weights.
In order to deal with the identifiability problem
let us introduce a distance between
two sets of parameters. Let
$\mathrm{dist}(\theta,\theta')$ denote the ordinary Euclidean distance (or any
other equivalent distance) between
two parameter vectors $\theta,\theta' \in \Theta$.
For $U,V \subset \Theta$ define
\begin{eqnarray}
 \mathrm{dist}(U,V)
  \equiv
  \inf_{\theta \in U} \inf_{\theta' \in V} \mathrm{dist}(\theta,\theta') .
  \nonumber
\end{eqnarray}
For a parameter $\theta$, let
\begin{eqnarray}
 \Theta(\theta)
  \equiv
  \{\theta' \in \Theta \mid f(x;\theta') = f(x;\theta)
  \quad
  \forall
  x\} .
  \nonumber
\end{eqnarray}
Then $\Theta_{0} = \Theta(\theta_{0})$ denotes the set of true parameters.
Since our densities are continuous with respect to $\theta$,
by Scheff\'e's theorem~(Theorem 16.12  of \cite{B1995})
$\mathrm{dist}(\Theta(\hat\theta_n), \Theta_0) \rightarrow 0$  implies
$\normlr{\hat{f}_{n}-f_{0}} \rightarrow 0$.

\section{Main results}
\label{sec:main}

We assume the following regularity conditions for
strong consistency of MLE.
\begin{assumption}
 There exist real constants
 $v_{0},v_{1} > 0$ and $\beta > 1$ such that
 \begin{eqnarray}
  f_{m}(x;\mu_{m}=0, \sigma_{m}=1)
   \leq
   \min \{v_{0} \;, \; v_{1} \cdot \abs{x}^{-\beta}\}
   \nonumber
 \end{eqnarray}
 for all $m$.
\label{assumption:1}
\end{assumption}

This assumption means that $f_{m} \; (m=1,\ldots, M)$ are bounded and
their tails decrease to zero faster than or equal to
$\abs{x}^{-\beta}$, which is a very mild condition.

The following three regularity conditions are standard
conditions assumed in discussing strong consistency of MLE.
Let $\Gamma$ denote any compact subset of $\Theta$.

\begin{assumption}
 For $\theta \in \Theta$ and any positive real number $\rho$, let
 \begin{eqnarray}
  f(x;\theta,\rho)
   & \equiv & \sup_{\mathrm{dist}(\theta',\theta) \leq \rho}f(x;\theta')
   .
   \nonumber
 \end{eqnarray}
 For each $\theta \in \Gamma$ and sufficiently small $\rho$,
 $f(x;\theta, \rho)$ is measurable.
\label{assumption:2}
\end{assumption}
\begin{assumption}
 For each $\theta \in \Gamma$,  if\/
 $\lim_{j \rightarrow \infty}\theta^{(j)} = \theta, \; (\theta^{(j)} \in \Gamma)$
 then\/
 $\lim_{j \rightarrow \infty}
 f(x;\theta^{(j)}) = f(x;\theta)$
 except on a set which is a null set and
 does not depend on the sequence
 $\{\theta^{(j)}\}_{j=1}^{\infty}$.
\label{assumption:3}
\end{assumption}
\begin{assumption}
 \begin{eqnarray}
  \int \abslr{
   \log{f(x;\theta_0)}
   }
   f(x;\theta_0)\mathrm{d} x < \infty .
   \nonumber
 \end{eqnarray}
\label{assumption:4}
\end{assumption}

Let
$E_{0}[\cdot]$ denote the expectation under the true parameter $\theta_{0}$.
The following theorem is essential to our
argument and it is of some independent interest.
\begin{thm}
 \label{kappa-lambda:thm}
 Suppose that assumptions \ref{assumption:1}--\ref{assumption:4}
 are satisfied
 and $f_{0} \in \mathscr{G}_{M} \backslash \mathscr{G}_{M-1}$
 where $\mathscr{G}_{M}$ and $\mathscr{G}_{M-1}$ are defined in
$(\ref{eq:def:scrg_K})$.
Then there  exist real constants $\kappa, \lambda > 0$ such that
 \begin{eqnarray}
  E_{0}
  \left[
   \log{
   \left\{
    g(x) + \kappa
   \right\}
   }
  \right]
  +
  \lambda
  <
  E_{0}[\log{f(x;\theta_{0})}]
  \label{kappa-lambda:eq}
 \end{eqnarray}
 for all
 $g \in \mathscr{G}_{M-1}$.
\end{thm}

We now state the main theorem of this paper.
\begin{thm}
 Suppose that assumptions \ref{assumption:1}--\ref{assumption:4}
 are satisfied
 and $f_{0} \in \mathscr{G}_{M}\backslash \mathscr{G}_{M-1}$
 where $\mathscr{G}_{M}$ and $\mathscr{G}_{M-1}$ are defined in
$(\ref{eq:def:scrg_K})$.
 Let $c_{0} > 0$ and $ 0 < d < 1$.
 If $c_{n} = c_{0} \cdot \exp(-n^{d})$ and
 \begin{eqnarray}
  \Theta_{n}
   \equiv
   \{
   \theta \in \Theta
   \mid
   \sigma_{m} \geq c_{n}\; , \; (m=1,\ldots,M)
   \},
   \nonumber
 \end{eqnarray}
 then
  \begin{eqnarray}
  \mathrm{Prob}
   \left(
    \lim_{n \rightarrow \infty}
    \mathrm{dist}(\Theta(\hat{\theta}_{n}), \Theta_0) = 0
   \right) = 1 \; ,
   \nonumber
\end{eqnarray}
where $\hat{\theta}_{n}$ is MLE
restricted to $\Theta_{n}$.
 \label{main-thm:thm}
\end{thm}

As remarked at the end of the previous section
theorem \ref{main-thm:thm} implies
the following corollary.
\begin{cor}
Under the same assumptions of $theorem~\ref{main-thm:thm}$,
$\hat{f}_{n}$ is strongly consistent
in the sense of definition~\ref{definition:1}.
\end{cor}

\section{Proofs}
\label{sec:proof}

In this section, we prove theorems stated in section \ref{sec:main}.
The organization of this section is as follows.  First in
subsection~\ref{sec:notat-some-lemm} we state some lemmas for
theorem~\ref{kappa-lambda:thm} and \ref{main-thm:thm}.  Next
in subsection~\ref{sec:proof-kappa-lambda-thm}
we prove
theorem~\ref{kappa-lambda:thm}
which is also essential for theorem~\ref{main-thm:thm}.
Finally we prove theorem~\ref{main-thm:thm}
in subsection~\ref{sec:proof-main-thm}.
For convenience a list of
notations used in our proofs is provided at the end of this paper.

\subsection{Notations and some lemmas}
\label{sec:notat-some-lemm}

Fix arbitrary $\kappa_0> 0$, which corresponds to $\kappa$ in theorem
\ref{kappa-lambda:thm}.
Define
$\tilde{\beta}$ and  $\nu(y)$, $y>0$,  as
\begin{eqnarray}
 \tilde{\beta}
  \equiv
  \frac{\beta-1}{\beta}
  \quad , \quad
 \nu(y) \equiv
  \left(
   \frac{v_{1}}{\kappa_{0}}
  \right)^{\frac{1}{\beta}}
  y^{\tilde{\beta}}
\ ,
 \label{eq:def:beta_and_nu_and_v}
\end{eqnarray}
where
$v_1$ and $\beta$ are given in assumption~\ref{assumption:1}.
Noting that $v_1 \cdot (\nu(y))^{-\beta} = \kappa_0 /y$,
the following lemma is easily proved and we omit its proof.
See figure \ref{fig:BoundedByStepFunction}.

\begin{lem}
\label{lem:BoundComponentByStepFunc}
Under the assumption~\ref{assumption:1},
for arbitrary $\kappa_0> 0$
each component density
$f_{m}(x; \mu, \sigma)$ is bounded by a step function
\begin{eqnarray}
 f_{m}(x; \mu, \sigma)
  \leq
  \max\{\mathrm{1}_{[\mu-\nu(\sigma), \mu+\nu(\sigma))}(x) \cdot \frac{v_0}{\sigma}
  \; , \; {\kappa_{0}}\}
  \leq
  \mathrm{1}_{[\mu-\nu(\sigma), \mu+\nu(\sigma))}(x)
  \cdot
  \frac{v_0}{\sigma}
  +
  {\kappa_{0}},
  \nonumber
\end{eqnarray}
 where $\mathrm{1}_{U}(x)$ denotes the indicator function of
$U \subset \mathbb{R}$.
\end{lem}

\begin{figure}[htbp]
 \begin{center}
  \includegraphics{b.eps}
  \caption{Each component is bounded by step function.}
  \label{fig:BoundedByStepFunction}
 \end{center}
\end{figure}

From lemma~\ref{lem:BoundComponentByStepFunc}
\begin{eqnarray}
  \sum_{m=1}^{M}
 \alpha_{m} f_{m}(x; \mu_{m}, \sigma_{m})
 & \leq &
  \sum_{m=1}^{M}
  \mathrm{1}_{[\mu_{m}-\nu(\sigma_{m}),
    \mu_{m}+\nu(\sigma_{m}))}(x)
    \cdot
\frac{v_0}{\sigma_{m}}
  +
  {\kappa_{0}}.
  \label{bounded-by-uniform-density:eq}
\end{eqnarray}
The right-hand side of (\ref{bounded-by-uniform-density:eq}) is a step function.
We look at this step function where the density $f(x;\theta)$ is high,
i.e.\ the scale parameter of some component is small.

For a given choice of $\kappa_0>0$, choose $c_0 > 0$ such that
\begin{equation}
 {c_0} < \frac{v_{0}}{\kappa_0(M+1)} \; ,
  \label{eq:kappa-c0-global-condition}
\end{equation}
Below  we will impose
additional constraints on $\kappa_{0}$ and $c_{0}$ to make
$\kappa_{0}$ and $c_{0}$ sufficiently small to satisfy other conditions.
For each $\theta$, let
\[
 \mathscr{K}_{{\sigma} \leq c_{0}} = \mathscr{K}_{{\sigma} \leq c_{0}}(\theta)
  \equiv
  \{m \mid 1 \leq m \leq M \; , \; \sigma_{m} \leq c_{0}\}
  \nonumber
\]
denote the set of components with $\sigma_m \le c_0$ and define
\begin{equation}
 J(\theta)
  \equiv
 \bigcup_{m \in \mathscr{K}_{{\sigma} \leq c_{0}}}
 [\mu_{m} - \nu(\sigma_{m}), \mu_{m} + \nu(\sigma_{m})).
 \label{eq:def:Jtheta}
\end{equation}
On $J(\theta)$ the density $f(x;\theta)$ is high.  Now dividing
$J(\theta)$ according to the height of the step function on the right-hand side of
(\ref{bounded-by-uniform-density:eq}), for $x \in J(\theta)$
we can write the right-hand side of
(\ref{bounded-by-uniform-density:eq}) as
\begin{equation}
 \mathrm{1}_{J(\theta)}(x) \cdot
\left\{
 \sum_{m=1}^{M}
  \mathrm{1}_{[{\mu}_{m}-\nu({\sigma}_{m}), {\mu}_{m}
    +\nu({\sigma}_{m}))}(x) \cdot
   \frac{v_0}{{\sigma}_{m}}
  +
  {\kappa_{0}}
  \right\}
  =
  \sum_{t=1}^{T(\theta)}
  H(J_{t}(\theta)) \cdot \mathrm{1}_{J_{t}(\theta)}(x),
  \nonumber
\end{equation}
where $J_{t}(\theta) \; (t=1,\ldots,T(\theta))$ are disjoint
intervals, $[\mu_{m}-\nu(\sigma_{m}),
\mu_{m}+\nu(\sigma_{m}))\; \; (m \in \mathscr{K}_{{\sigma} \leq
  c_{0}})$ are unions of some of $J_{t}(\theta)$'s and the height
$H(J_{t}(\theta))$ for each $t$ is defined by any $x \in
J_{t}(\theta)$ as
\begin{eqnarray}
 H(J_{t}(\theta)) \equiv
  \sum_{m=1}^{M} \mathrm{1}_{[\mu_{m}-\nu(\sigma_{m}),
    \mu_{m}+\nu(\sigma_{m}))}(x) \cdot
  \frac{v_0}{\sigma_{m}}
  + {\kappa_{0}}.
\label{eq:def:HJtn}
\end{eqnarray}
See figure \ref{fig:J_t}.  For $x\in J_t(\theta)$, there is at least
one $m=m_t$ such that
$x\in [\mu_{m}-\nu(\sigma_{m}),
\mu_{m}+\nu(\sigma_{m}))$ and
$H(J_{t}(\theta))\ge v_0/c_0+\kappa_0$.
\begin{figure}[htbp]
 \begin{center}
  \includegraphics{J_t.eps}
  \caption{Definition of $J_{t}(\theta)$.}
  \label{fig:J_t}
 \end{center}
\end{figure}
Also note that the total number $T(\theta)$ of $J_{t}(\theta)$'s satisfies
$T(\theta) \le 2M$, because the change of the height can only occur at
$\mu_{m}-\nu(\sigma_{m})$ or $\mu_{m}+\nu(\sigma_{m})$.

By   (\ref{bounded-by-uniform-density:eq})
we have the following lemma for $x\in J(\theta)$.
\begin{lem}
\label{lem:BoundMixtureByStepFunc}
Under the assumption~\ref{assumption:1}, for each  $x \in J(\theta)$
\[
  \sum_{m=1}^{M}
  \alpha_{m} f_{m}(x; \mu_{m}, \sigma_{m})
  \leq
  \sum_{t=1}^{T(\theta)}
  H(J_{t}(\theta)) \cdot \mathrm{1}_{J_{t}(\theta)}(x).
\]
\end{lem}

A density can be high only in a small region and we want to have some
explicit bound on the length $W(J_{t}(\theta))$ of $J_{t}(\theta)$
in terms of its height $H(J_{t}(\theta))$.
Let
\begin{equation}
   v_{2}
   \equiv
   2\left(
     \frac{v_{1}}{\kappa_{0}}
    \right)^{\frac{1}{\beta}}
   \left(
    {v_{0}\cdot {(M + 1)}}
   \right)^{\tilde{\beta}}
  \quad , \quad
  \xi(y) \equiv v_{2}\cdot \left(\frac{1}{y}\right)^{\tilde{\beta}}, \ y>0
  \ ,
  \label{eq:def:v_2_and_xi}
\end{equation}
where $v_0, v_1$ and $\beta$ are given in assumption~\ref{assumption:1}
and $\tilde \beta$ is defined in  (\ref{eq:def:beta_and_nu_and_v}).
\begin{lem}
\label{lem:BoundWidthOfInterval}
Under the assumption~\ref{assumption:1}, the length $W(J_{t}(\theta))$
of $J_{t}(\theta)$ for each $t$ is bounded as
 \begin{equation}
  W(J_{t}(\theta))
   \leq
   v_{2} \cdot
   \left(
    \frac{1}{{H(J_{t}(\theta))}}
   \right)^{\tilde{\beta}}
   = \xi(H(J_{t}(\theta)))
   \ .
   \nonumber
 \end{equation}
\end{lem}
\noindent {\bf Proof: }\
In (\ref{eq:def:HJtn}) at any $x\in J_{t}(\theta)$,
$H(J_{t}(\theta)) - \kappa_{0}$
consists of at most $M$ components.
Thus, for each $J_{t}(\theta)$, there exists at least one component $m=m_t$ such that
\begin{eqnarray}
\frac{v_0}{\sigma_{m}}
  \geq
  \frac{1}{M}
  \cdot
  \left(
   H(J_{t}(\theta)) -{\kappa_{0}}
  \right) \; .
  \nonumber
\end{eqnarray}
Furthermore from (\ref{eq:kappa-c0-global-condition}) we have
\begin{eqnarray}
  H(J_{t}(\theta)) - {\kappa_{0}}
  & > &
  H(J_{t}(\theta)) - \frac{v_{0}}{c_{0}(M + 1)}
  \nonumber \\
  & > &
  H(J_{t}(\theta)) -  \frac{H(J_{t}(\theta))}{M + 1}
  =
  \frac{M \cdot H(J_{t}(\theta))}{M + 1}
  \nonumber
  .
\end{eqnarray}
Therefore we have
\[
  \nu(\sigma_{m})
=
  \left(
   \frac{v_{1}}{\kappa_{0}}
  \right)^{\frac{1}{\beta}}
  \sigma_{m}^{\tilde\beta}
\leq
  \left(
   \frac{v_{1}}{\kappa_{0}}
  \right)^{\frac{1}{\beta}}
  \left(
   \frac{v_{0}{(M + 1)}}{{H(J_{t}(\theta))}}
  \right)
  ^{ \tilde\beta
  }  \ .
\]
This with $W(J_{t}(\theta)) \leq 2\nu(\sigma_{m})$
proves the lemma.
\qed
\bigskip

So far we have been concerned with bounding the density at its peaks.
Now we consider bounding the tail of the true density $f(x;\theta_0)$.
Write
$\bar{\mu}_{0} \equiv \max(|\mu_{01}|, \ldots, |\mu_{0M}|)$
and
$
\theta_0=(\alpha_{01},\mu_{01},\sigma_{01}, \ldots, \alpha_{0M},
\mu_{0M}, \sigma_{0M}).
$
Let
\begin{equation}
 u_0 \equiv \sup_x f(x;\theta_0)
  \quad , \quad
  u_1 \equiv \max( u_0 \cdot (2 \bar{\mu}_{0})^{\beta} , \;
  2^{\beta} v_1 \sum_{m=1}^M \alpha_{0m}
  \sigma_{0m}^{\beta-1})
  \; .
  \label{eq:def:u_0_and_u_1}
\end{equation}
\begin{lem}
\label{lem:true-tail}
Under the assumption~\ref{assumption:1},
the following inequality holds.
\begin{eqnarray}
 f(x;\theta_{0})
  \leq
  \min{\{u_{0}, \; u_{1}\cdot \abs{x}^{-\beta}\}}, \quad \forall x\in \mathbb{R}.
  \nonumber
\end{eqnarray}
\end{lem}
\noindent {\bf Proof: }\
From assumption~\ref{assumption:1}
$$
u_0 \le \sum_{m=1}^M \alpha_{0m} \frac{v_0}{\sigma_{0m}}.
$$
Then for $|x| \ge 2\bar{\mu}_{0}$
$$
|x-\mu_{0m}|^{-\beta} \le (|x| - \bar{\mu}_{0})^{-\beta}
\le 2^\beta |x|^{-\beta} \ ,\quad  (m=1,\ldots,M).
$$
Therefore for $|x| \ge 2\bar{\mu}_{0}$
$$
f(x;\theta_0) \le |x|^{-\beta} 2^\beta v_1 \sum_{m=1}^M \alpha_{0m}
\sigma_{0m}^{\beta-1}
$$
and
\begin{eqnarray}
 f(x;\theta_{0})
  \leq
  \min{\{u_{0}, u_{1}\cdot \abs{x}^{-\beta}\}}, \quad \forall x\in \mathbb{R}.
  \nonumber
\end{eqnarray}
\qed
\bigskip

Based on lemma \ref{lem:true-tail} we can bound
the behavior of the minimum and the maximum of the sample.
Let  $x_{1}, \ldots, x_{n}$ denote a random sample
of size $n$ from $f(x;\theta_{0})$ and let
\begin{eqnarray}
 x_{n,1}  \equiv  \min{\{x_{1}, \ldots, x_{n}\}}
  \quad , \quad
 x_{n,n}  \equiv  \max{\{x_{1}, \ldots, x_{n}\}}.
  \nonumber
\end{eqnarray}
The following lemma follows from the Borel-Cantelli lemma.
\begin{lem}
 \label{expanding-A:lem}
For any real constant $A_{0} > 0$ and $\zeta > 0$,
define
\begin{eqnarray}
 A_{n} \equiv A_{0} \cdot n^{\frac{2 + \zeta}{\beta - 1}}.
  \nonumber
\end{eqnarray}
Then
\begin{eqnarray}
 \mathrm{Prob}
  \left(
   x_{n,1} < -A_{n}
   \quad \mathrm{or} \quad
   x_{n,n} > A_{n}
   \quad i.o.
  \right)
  = 0 \ .
  \nonumber
\end{eqnarray}
\end{lem}
\noindent {\bf Proof: }\
By the Borel-Cantelli lemma and the Bonferroni inequality
it suffices to show that
\[
 \sum_{n=1}^{\infty}
  \mathrm{Prob}
  \left(
   x_{n,1} < -A_{n}
  \right)
 <
 \infty ,  \qquad
 \sum_{n=1}^{\infty}
 \mathrm{Prob}
  \left(
   x_{n,n} > A_{n}
  \right)
  <
 \infty .
\]
We consider the left tail. Let $F_{0}(x)$ denotes the distribution
function of $f(x;\theta_{0})$.
Then
\begin{eqnarray}
 \mathrm{Prob}
  \left(
   x_{n,1} < -A_{n}
  \right)
 & = &
 1 - (1 - F_{0}(-A_{n}))^{n} ,
   \nonumber
\end{eqnarray}
and
\begin{eqnarray}
 F_{0}(-A_{n})
  & \leq &
  \int_{-\infty}^{-A_{n}}
  u_{1} \cdot \abs{x}^{-\beta}
  \mathrm{d} x
  =
  \frac{u_{1}}{\beta - 1}
  \cdot A_{n}^{-\beta + 1} .
   \nonumber
\end{eqnarray}
By replacing $n$ by $n-n_0$ with a  sufficiently large $n_0$ if necessary,
we can assume without loss
of generality that
\begin{eqnarray}
 \frac{u_{1}A_{0}^{-\beta + 1}}{\beta - 1}
  \left(
   n^{\frac{2 + \zeta}{\beta - 1}}
  \right)^{-\beta + 1} < 1, \quad \forall n.
  \nonumber
\end{eqnarray}
Then
\begin{eqnarray}
 \log  (1 - F_{0}(-A_{n}))^{n}
  & \geq &
  \log{
   \left(
    1 - \frac{u_{1}}{\beta - 1}\cdot A_{n}^{-\beta + 1}
   \right)^{n}
  }
  \nonumber \\
  & = &
   \log{
    \left(
     1 - \frac{u_{1}A_{0}^{-\beta + 1}}{\beta - 1}
     \left(
      n^{\frac{2 + \zeta}{\beta - 1}}
     \right)^{-\beta + 1}
    \right)^{n}
   } .
   \nonumber
\end{eqnarray}
Let
$u_{2} \equiv u_{1}A_{0}^{-\beta + 1}/(\beta - 1)$
and we have
\begin{eqnarray}
 \abslr{
  \log{
  (1 - F_{0}(-A_{n}))^{n}
  }}
 & \leq &
  \abslr{
  \log{
  \left(1 - \frac{u_{2}}{n^{2 + \zeta}}\right)^{n}
  }}
  \nonumber \\
 & = &
 \frac{u_{2}}{n^{1 + \zeta}}
 \abslr{
 \log{
 \left(
  1 - \frac{u_{2}}{n^{2 + \zeta}}
 \right)^{\frac{n^{2 + \zeta}}{u_{2}}}
 }}
  \nonumber \\
 & = &
  O(n^{-(1 + \zeta)}).
  \nonumber
\end{eqnarray}
Hence there exists a  sufficiently large
$N$ and $u_{3} > 0$
such that
\begin{eqnarray}
  \abslr{
   \log{
  (1 - F_{0}(-A_{n}))^{n}
  }}
  \leq
  \frac{u_{3}}{n^{1 + \frac{\zeta}{2}}}
  \nonumber
\end{eqnarray}
for all $n > N$.
This and
$(1 - F_{0}(-A_{n}))^{n} \leq 1$ imply that  for $n > N$
\begin{eqnarray}
  \log{
  (1 - F_{0}(-A_{n}))^{n}
  }
  \geq
  -\frac{u_{3}}{n^{1 + \frac{\zeta}{2}}}.
  \nonumber
\end{eqnarray}
Hence by $1-e^{-y} \leq y$, we have  for $n>N$
\begin{eqnarray}
 \mathrm{Prob}(x_{n,1} < -A_{n})
  =
 1 - (1 - F_{0}(-A_{n}))^{n}
  \leq
 1 - \exp{\left(-\frac{u_{3}}{n^{1 + \frac{\zeta}{2}}}\right)}
 \leq
 \frac{u_{3}}{n^{1 + \frac{\zeta}{2}}}.
 \nonumber
\end{eqnarray}
Therefore we obtain
\begin{eqnarray}
 \sum_{n > N}
  \mathrm{Prob}(x_{n,1} < -A_{n})
  =
  \sum_{n > N} 1 - (1 - F_{0}(-A_{n}))^{n}
 \leq
 \sum_{n} \frac{u_{3}}{n^{1 + \frac{\zeta}{2}}}
 < \infty .
  \nonumber
\end{eqnarray}
The case of  the right tail $\mathrm{Prob}(x_{n,n} > A_{n})$ is also proved
by the same argument.
\qed
\bigskip

Finally we consider subprobability densities in $\mathscr{G}_{\mathscr{K}}$.
For any positive real number $\rho$, let
\begin{eqnarray}
 f_{\mathscr{K}}(x;\theta_{\mathscr{K}},\rho)
 & \equiv &
 \sup_{\mathrm{dist}(\theta_{\mathscr{K}}',\theta_{\mathscr{K}}) \leq \rho}f_{\mathscr{K}}(x;\theta_{\mathscr{K}}')
 \quad , \quad
 (\theta_{\mathscr{K}}' \in  \bar{\Theta}_{\mathscr{K}}) .
 \label{eq:def:f_scrk_rho}
\end{eqnarray}
The following lemma follows from the bounded convergence theorem.
\begin{lem}
 \label{expectation-equality:lem}
 Let
 ${\Gamma}_{\mathscr{K}}$
 denote any compact subset of $ \bar{\Theta}_{\mathscr{K}}$.
 For any real constant $\kappa \geq 0$ and
 any point $\theta_{\mathscr{K}} \in {\Gamma}_{\mathscr{K}}$,
 the following equality holds
 under the assumption~\ref{assumption:1} and \ref{assumption:3}.
 \begin{eqnarray}
  \lim_{\rho \rightarrow 0}
   E_{0}[\log\{f_{\mathscr{K}}(x;\theta_{\mathscr{K}}, \rho) + \kappa\}]
    =
    E_{0}[\log\{f_{\mathscr{K}}(x;\theta_{\mathscr{K}}) + \kappa\}] \ .
    \nonumber
 \end{eqnarray}
\end{lem}
\noindent {\bf Proof: }\
We treat the case of $\kappa > 0$.
The case of $\kappa = 0$ is almost the same as
the proof of Lemma 2 in \cite{W1949}.
{}From assumption  \ref{assumption:3} we have
\begin{eqnarray}
 \lim_{\rho \rightarrow 0}
  \log\{f_{\mathscr{K}}(x;\theta_{\mathscr{K}}, \rho) + \kappa\}
  =
  \log\{f_{\mathscr{K}}(x;\theta_{\mathscr{K}}) + \kappa\}
  \qquad a.e.
  \nonumber
\end{eqnarray}
Now
${\Gamma}_{\mathscr{K}}$
is compact and $\kappa > 0$.
Hence by assumption  \ref{assumption:1},
$\log\{f_{\mathscr{K}}(x;\theta_{\mathscr{K}}, \rho) + \kappa\}$
is bounded.
Therefore
\begin{eqnarray}
 \lim_{\rho \rightarrow 0}
  E_{0}[\log\{f_{\mathscr{K}}(x;\theta_{\mathscr{K}}, \rho) + \kappa\}]
  =
  E_{0}[\log\{f_{\mathscr{K}}(x;\theta_{\mathscr{K}}) + \kappa\}]
  \nonumber
\end{eqnarray}
by the bounded convergence theorem.
\qed
\bigskip

\subsection{Proof of theorem~\ref{kappa-lambda:thm}}
\label{sec:proof-kappa-lambda-thm}

In this section we prove theorem~\ref{kappa-lambda:thm} by
contradiction.  Fix arbitrary proper subset $\mathscr{L}$ of $\{1,\dots , M\}$.
It suffices to prove that (\ref{kappa-lambda:eq}) holds for
all $g \in \mathscr{G}_{\mathscr{L}}$.
Suppose that~(\ref{kappa-lambda:eq}) does not hold for some
$\mathscr{G}_{\mathscr{L}}$.
Then  for any $\lambda, \kappa > 0$,
there exists $g \in \mathscr{G}_{\mathscr{L}}$ such that
\begin{eqnarray}
 E_{0}[\log{\{g(x) + \kappa\}}] + \lambda \geq E_{0}[\log{f(x;\theta_{0})}].
  \nonumber
\end{eqnarray}
Here, let $\{\lambda_{j}\}, \{\kappa_{j}\}$
be positive sequences which decrease
to zero.
Then for each $\lambda_{j}, \kappa_{j} > 0$,
there exists $g_{j} \in \mathscr{G}_{\mathscr{L}}$ such that
\begin{eqnarray}
 E_{0}[\log{\{g_{j}(x) + \kappa_{j}\}}] + \lambda_{j}
  \geq E_{0}[\log{f(x;\theta_{0})}] \; .
  \nonumber
\end{eqnarray}
It follows that
\begin{eqnarray}
 \liminf_{j \rightarrow \infty}
 E_{0}[\log{\{g_{j}(x) + \kappa_{j}\}}] + \lambda_{j}
  \geq E_{0}[\log{f(x;\theta_{0})}] \; .
  \label{eq:contradictory_assumption}
\end{eqnarray}
Now $g_{j}$ can be written as
\begin{eqnarray}
 g_{j}(x) = f_{\mathscr{L}}(x; \theta_{\mathscr{L}}^{(j)}) \; .
  \nonumber
\end{eqnarray}
Then the following lemma holds by compactification argument.
\begin{lem}
\label{lem:proof-theor-refk}
 There exists a subsequence of
 $\{\theta_{\mathscr{L}}^{(j)}\}_{j=1}^{\infty} \equiv \{\{\alpha_{m}^{(j)}, \mu_{m}^{(j)}, \sigma_{m}^{(j)} \mid m \in \mathscr{L}\}\}_{j=1}^{\infty}$
 and disjoint subsets
 $
 \mathscr{K}_{{\sigma}\downarrow 0},
 \mathscr{K}_{{\sigma}\uparrow \infty},
 \mathscr{K}_{\abs{\mu}\uparrow \infty}
 \subset \mathscr{L}
 $
such that along the subsequence:
\begin{align*}
&\sigma_{m}^{(j)} \rightarrow 0 \text{\quad for\quad} m \in
\mathscr{K}_{{\sigma}\downarrow 0}, \\
&\sigma_{m}^{(j)} \rightarrow \infty
\text{\quad for\quad} m \in \mathscr{K}_{{\sigma}\uparrow \infty},\\
&\sigma_{m}^{(j)} \text{ converges to a finite value and }
\abs{\mu_{m}^{(j)}} \rightarrow \infty \text{\ for\ }
m \in \mathscr{K}_{\abs{\mu}\uparrow \infty},\\
&(\alpha_{{m}}^{(j)}, \mu_{{m}}^{(j)} ,
\sigma_{{m}}^{(j)}) \text{ converges to  a finite point }
(\alpha_{{m}}^{(\infty)}, \mu_{{m}}^{(\infty)} , \sigma_{{m}}^{(\infty)})
 \text{ for } m \in \scrk_{{R}},
\end{align*}
 where $\scrk_{{R}} \equiv \mathscr{L}
 \backslash \{\mathscr{K}_{{\sigma}\downarrow 0} \cup \mathscr{K}_{{\sigma}\uparrow \infty}
 \cup \mathscr{K}_{\abs{\mu}\uparrow \infty}\}$.
\end{lem}
\noindent {\bf Proof: }\
Let
\begin{eqnarray}
 {\mu'}_{m}^{(j)} \equiv \arctan{(\mu_{m}^{(j)})},
\qquad
 {\sigma'}_{m}^{(j)} \equiv \arctan{(\sigma_{m}^{(j)})}.
  \nonumber
\end{eqnarray}
Then
$\{\{\alpha_{m}^{(j)}, {\mu'}_{m}^{(j)}, {\sigma'}_{m}^{(j)} \mid m \in \mathscr{L}\}\}_{j=1}^{\infty}$
is regarded as a sequence in the following compact set.
\begin{equation}
 0 \leq \alpha_{m} \leq 1
  \ , \
  \sum_{m = 1}^{L} \alpha_{m} \leq 1 \ ,\
 -\frac{\pi}{2} \leq {\mu'}_{m}^{(j)} \leq \frac{\pi}{2} \ , \
 0 \leq {\sigma'}_{m}^{(j)} \leq \frac{\pi}{2}.
 \label{extended-parameter-space:eq}
\end{equation}
Therefore there exists a subsequence of
$\{\{\alpha_{m}^{(j)}, {\mu'}_{m}^{(j)}, {\sigma'}_{m}^{(j)} \mid m \in \mathscr{L}\}\}_{j=1}^{\infty}$
that converges to a point in the set (\ref{extended-parameter-space:eq}).
Now we sort the elements in $\mathscr{L}$ according to their behaviors in this subsequence.
First, we choose components such that
$\sigma_{m}^{(j)} \rightarrow 0$,
and add these $m$ to $\mathscr{K}_{{\sigma}\downarrow 0}$.
Second, from the remainder,
we choose components such that
$\sigma_{m}^{(j)} \rightarrow \infty$,
and add these $m$ to $\mathscr{K}_{{\sigma}\uparrow \infty}$.
Third, from the remainder,
we choose components such that
$\abs{\mu_{m}^{(j)}} \rightarrow \infty$
and add these $m$ to $\mathscr{K}_{\abs{\mu}\uparrow \infty}$.
Finally, we choose
the remaining components
as $\scrk_{{R}}$.
\qed

\bigskip
From lemma~\ref{lem:proof-theor-refk}, we define $g_{\infty}$ as follows.
\begin{eqnarray}
 g_{\infty}(x)
  \equiv
  \sum_{m \in \scrk_{{R}}}
  {\alpha}_{m}^{(\infty)} f_{m}(x; {\mu}_{m}^{(\infty)}, {\sigma}_{m}^{(\infty)})
  \in \mathscr{G}_{\scrk_{{R}}} \; .
  \nonumber
\end{eqnarray}
For notational simplicity and without loss of generality,
we replace the original sequence with this subsequence, because
(\ref{eq:contradictory_assumption}) holds for this subsequence as well.
Furthermore, by considering the sequence
$\{\theta_{\mathscr{L}}^{(j)}\}_{j=j_{0}}^{\infty}$
where $j_{0}$ is sufficiently large and
replacing $j$ by $j-j_0$ if necessary,
we can assume without loss
of generality that there exist
sufficiently small real constants
$\kappa_{0} > 0$ and $c_{0} > 0$
such that
\begin{eqnarray}
 & &
  E_{0}[\log{f(x;\theta_{0})}]
  -
  E_{0}
  \left[
   \log{
   \left\{
    g_{\infty}(x) + 3\kappa_{0}
    \right\}
   }
  \right]
  >
  0
  \;\; , \;\;
  {\kappa_{0}}
  <
  \frac{v_0}{c_0(M + 1)}
  \; , \nonumber \\
 & &
  \sigma_{m}^{(j)} < c_{0}
  \;\; (m \in \mathscr{K}_{{\sigma}\downarrow 0})
  \;\; , \;\;
  \sigma_{m}^{(j)} > \frac{v_{0}}{\kappa_{0}}
  \;\  (m \in \mathscr{K}_{{\sigma}\uparrow \infty})
  \; , \nonumber \\
 & &
  c_{0} \leq \sigma_{m}^{(j)} \leq \frac{v_{0}}{\kappa_{0}}
  \;\ (m \in \mathscr{K}_{\abs{\mu}\uparrow \infty})
  \quad \text{for all }j \; .
  \label{kappa-zero-and-c-zero:eq}
\end{eqnarray}
From lemma~\ref{lem:BoundComponentByStepFunc} and \ref{lem:BoundMixtureByStepFunc}, we have
\begin{eqnarray}
 \lefteqn{
  E_{0}
  \left[
   \log{
   \left\{
    f_{\mathscr{L}}(x; {\theta}_{\mathscr{L}}^{(j)})
    + \kappa_{j}
   \right\}
   }
   + \lambda_{j}
  \right]
  }
  \nonumber \\
 & \leq &
  \int
  \mathrm{1}_{J({\theta}_{\mathscr{L}}^{(j)})}(x)
  \cdot
  \log{
  \left\{
   \sum_{t=1}^{T({\theta}_{\mathscr{L}}^{(j)})}
   H(J_{t}({\theta}_{\mathscr{L}}^{(j)}))\cdot \mathrm{1}_{J_{t}({\theta}_{\mathscr{L}}^{(j)})}(x)
   + \kappa_{j}
  \right\}
  }
  f(x;\theta_{0})
  \mathrm{d} x
  \nonumber \\
 & & +
  \int
  \mathrm{1}_{\mathbb{R} \backslash J({\theta}_{\mathscr{L}}^{(j)})}(x)
  \cdot
  \log{
  \left\{
   f_{\mathscr{K}_{{\sigma} > 0}}(x; \theta_{\mathscr{K}_{{\sigma} > 0}}^{(j)})
   +
   {\kappa_{0}}
   + \kappa_{j}
  \right\}
  }
  f(x;\theta_{0})
  \mathrm{d} x
   + \lambda_{j} \; ,
   \nonumber \\
  \label{inequality-of-expectation-of-g-infty:eq}
\end{eqnarray}
where
$\mathscr{K}_{{\sigma} > 0} \equiv \mathscr{L}
 \backslash \mathscr{K}_{{\sigma}\downarrow 0}$.

Now we evaluate the first term
on the right-hand side of (\ref{inequality-of-expectation-of-g-infty:eq}).
{}From lemma \ref{lem:BoundWidthOfInterval}
\begin{eqnarray}
 \lefteqn{
  \int
  \mathrm{1}_{J({\theta}_{\mathscr{L}}^{(j)})}(x)
  \cdot
  \log{
  \left\{
   \sum_{t=1}^{T({\theta}_{\mathscr{L}}^{(j)})}
   H(J_{t}({\theta}_{\mathscr{L}}^{(j)})) \cdot \mathrm{1}_{J_{t}({\theta}_{\mathscr{L}}^{(j)})}(x)
   + \kappa_{j}
  \right\}
  }
  f(x;\theta_{0})
  \mathrm{d} x
  } & &
  \nonumber \\
 & \leq &
  \sum_{t=1}^{T({\theta}_{\mathscr{L}}^{(j)})}
  W(J_{t}({\theta}_{\mathscr{L}}^{(j)}))
  \cdot
  \log{
  \left\{
   H(J_{t}({\theta}_{\mathscr{L}}^{(j)}))
   + \kappa_{j}
  \right\}
  }
  \cdot
  u_{0}
 \longrightarrow 0
 \quad , \quad
  (n \rightarrow \infty)\ ,
  \label{jack:eq}
\end{eqnarray}
where $u_{0} = \sup_x f(x;\theta_0)$ defined in (\ref{eq:def:u_0_and_u_1}).
Next we evaluate the second term
on the right-hand side of (\ref{inequality-of-expectation-of-g-infty:eq}).
Let
\begin{eqnarray}
 A^{({j})}
  \equiv
 \min_{m \in \mathscr{K}_{\abs{\mu}\uparrow \infty}}
  \left\{
   \min{\{\abs{{\mu}_{m}^{(j)} + \nu({\sigma}_{m}^{(j)})} , \abs{{\mu}_{m}^{(j)} - \nu({\sigma}_{m}^{(j)})}\}}
  \right\}.
  \label{eq:def:A_hat_seqnum}
\end{eqnarray}
Then
\begin{eqnarray}
& &
f_{\mathscr{K}_{{\sigma} > 0}}(x; \theta_{\mathscr{K}_{{\sigma} > 0}}^{(j)})
 =
f_{\mathscr{K}_{{\sigma} \uparrow \infty}}
(x;\theta_{\mathscr{K}_{{\sigma} \uparrow \infty}}^{(j)})
+
f_{\mathscr{K}_{\abs{\mu} \uparrow \infty}}
(x;\theta_{\mathscr{K}_{\abs{\mu} \uparrow \infty}}^{(j)})
+
f_{\scrk_{{R}}}(x; \theta_{\scrk_{{R}}}^{(j)})
\; ,
\nonumber \\
& & f_{\mathscr{K}_{{\sigma} \uparrow \infty}}(x;\theta_{\mathscr{K}_{{\sigma} \uparrow \infty}}^{(j)}) \leq \kappa_{0}
\quad \text{ for all } x
 \; , \nonumber \\
& & f_{\mathscr{K}_{{\sigma} \uparrow \infty}}
(x;\theta_{\mathscr{K}_{{\sigma} \uparrow \infty}}^{(j)})
 + f_{\mathscr{K}_{\abs{\mu} \uparrow \infty}}
(x;\theta_{\mathscr{K}_{\abs{\mu} \uparrow \infty}}^{(j)})
 \leq \kappa_{0}
\quad \text{ for } x \in [-A^{({j})},A^{({j})}] \backslash J({\theta}_{\mathscr{L}}^{(j)}) \; .
\nonumber
\end{eqnarray}
Therefore the following inequality holds.
\begin{eqnarray}
 \lefteqn{
  \int
  \mathrm{1}_{\mathbb{R} \backslash J({\theta}_{\mathscr{L}}^{(j)})}(x)
  \cdot
  \log{
  \left\{
   f_{\mathscr{K}_{{\sigma} > 0}}(x; \theta_{\mathscr{K}_{{\sigma} > 0}}^{(j)})
   +
   {\kappa_{0}}
   + \kappa_{j}
  \right\}
  }
  f(x;\theta_{0})
  \mathrm{d} x
  } & &
  \nonumber \\
 & \leq &
  \int
  \mathrm{1}_{[-A^{({j})},A^{({j})}] \backslash J({\theta}_{\mathscr{L}}^{(j)})}(x)
  \cdot
  \log{
  \left\{
   f_{\scrk_{{R}}}(x; \theta_{\scrk_{{R}}}^{(j)})
   +
   {2\kappa_{0}}
   + \kappa_{j}
  \right\}
  }
  f(x;\theta_{0})
  \mathrm{d} x
  \nonumber \\
 & & +
  \int
  \mathrm{1}_{
  \{(-\infty, -A^{({j})}) \bigcup (A^{({j})}, \infty)\}
  \backslash J({\theta}_{\mathscr{L}}^{(j)})}(x)
  \cdot
  \log{
  \left\{
   f_{\mathscr{K}_{\abs{\mu}\uparrow \infty}}(x; \theta_{\mathscr{K}_{\abs{\mu}\uparrow \infty}}^{(j)})
  \right.
  }
  \nonumber \\
 & & \qquad\qquad
  { +
  \left.
   f_{\scrk_{{R}}}(x; \theta_{\scrk_{{R}}}^{(j)})
   +
   2{\kappa_{0}}
   + \kappa_{j}
  \right\}
  }
  f(x;\theta_{0})
  \mathrm{d} x
  \nonumber   \\
 &\equiv& I_1^{(j)} + I_2^{(j)} \qquad{\rm(say)} .
  \label{tree-of-beans:eq}
\end{eqnarray}
By the bounded convergence theorem,
we obtain
\begin{eqnarray}
I_1^{(j)}
  \rightarrow
  \int
  \log{
  \left\{
   g_{\infty}(x)
   +
   {2\kappa_{0}}
  \right\}
  }
  f(x;\theta_{0})
  \mathrm{d} x
  \quad , \quad
  I_2^{(j)}
   \rightarrow
  0.
  \label{fatboy-slim-2:eq}
\end{eqnarray}

{}From (\ref{inequality-of-expectation-of-g-infty:eq}),
(\ref{jack:eq}), (\ref{tree-of-beans:eq}),
(\ref{fatboy-slim-2:eq}),
we have
\begin{eqnarray}
 E_{0}[\log f(x;\theta_{0})]
  \leq
 \limsup_{j \rightarrow \infty}
  E_{0}[\log\{g_{j}(x) + \kappa_{j}\}] + \lambda_{j}
  \leq
  E_{0}
  \left[
   \log{
   \left\{
    g_{\infty}(x)
    +
    {2\kappa_{0}}
   \right\}
   }
  \right].
  \nonumber
\end{eqnarray}
This is a contradiction to (\ref{kappa-zero-and-c-zero:eq}).
This completes the proof of theorem \ref{kappa-lambda:thm}.

\subsection{Proof of the main theorem}
\label{sec:proof-main-thm}

We choose real constants $\kappa$ and $\lambda$ to satisfy (\ref{kappa-lambda:eq})
by using theorem \ref{kappa-lambda:thm}.
Having chosen these constants,
from now on we follow the line of the proof
in \cite{TT2003-20}, although the details of the proof here is much more
complicated.
For the sake of readability we divide our proof into further sections.

\subsubsection{Setting up constants}
For $\kappa, \lambda$
satisfying~(\ref{kappa-lambda:eq}), let $\kappa_{0}, \lambda_{0}$ be
real constants such that
\begin{equation}
 0 < 4\kappa_{0} \leq \kappa \quad , \quad 0 < 4\lambda_{0} \leq \lambda \; .
  \nonumber
\end{equation}
Note that $4\kappa_{0}, 4\lambda_{0}$ also satisfy~(\ref{kappa-lambda:eq}).
Define
\begin{eqnarray}
 B \equiv \frac{v_{0}}{\kappa_{0}}
  >
  \max{\{\sigma_{01}, \ldots, \sigma_{0M}\}}.
  \label{eq:def:B}
\end{eqnarray}
If $\sigma_{m} \geq B$, then the density of the $m$-th component is almost flat
and makes little contribution to the likelihood.
In section \ref{sec:part-param-space}, we partition the parameter space
according to this property.

Because $\{c_n\}$ is decreasing
to zero, by replacing $c_0$ by some $c_n$ if necessary,
we can assume without loss
of generality that $c_0$ is sufficiently small to satisfy the
following conditions,
\begin{eqnarray}
 & & (v_0/c_0)^{\tilde{\beta}}
  > e ,
  \nonumber \\
 & & c_{0}  <  \min{\{\sigma_{01}, \ldots, \sigma_{0M}\}} ,
  \nonumber \\
 & & 3M \cdot u_{0} \cdot 2\nu(c_{0}) \cdot \abslr{\log{ {\kappa_{0}} }}
   <
  {\lambda_{0}} ,
   \nonumber
  \\
 & &
 3 \cdot 2M \cdot u_{0} \cdot
  \xi(v_0/c_{0})
  \cdot \log(v_0/c_0)
   <
  {\lambda_{0}}
  \; ,
   \nonumber
  \\
  & & {\kappa_{0}}
  <
  \frac{v_0}{c_0(M + 1)}
   \quad ,
   \label{eq:condition:proof:mainthm}
\end{eqnarray}
where $\tilde{\beta}$, $\nu(\cdot)$
and $\xi(\cdot)$ are defined in
(\ref{eq:def:beta_and_nu_and_v}) and (\ref{eq:def:v_2_and_xi}).

For any subset $V\subset \mathbb{R}$, let $P_0(V)$ denote the probability of
$V$ under the true density
\begin{eqnarray}
 P_0(V) \equiv \int_{V} f(x;\theta_{0}) \mathrm{d} x \; .
  \label{eq:def:P_0}
\end{eqnarray}
Let $A_{0} > 0$ be a positive constant which satisfies
\begin{eqnarray}
  P_0(\mathscr{A}_{0})
  \cdot
  \log{
  \left(
   \frac{v_0/c_0 + 2\kappa_{0}}{3\kappa_{0}}
  \right)
  }
  < {\lambda_{0}} ,
  \label{A-zero:condition:eq}
\end{eqnarray}
where
\begin{equation}
 \mathscr{A}_{0} \equiv (-\infty, -A_{0}] \cup [A_{0},\infty).
  \label{eq:def:scra_0}
\end{equation}
Let
$
 A_{n} \equiv A_{0} \cdot n^{\frac{2 + \zeta}{\beta - 1}}
  \nonumber
$
as in lemma \ref{expanding-A:lem}.
Define a subset $\Theta_{n}'$ of $\Theta_{n}$ in
theorem~\ref{main-thm:thm} by
\begin{eqnarray}
  \Theta_{n}'
  \equiv
  \{
  \theta \in \Theta_{n}
  \mid
  \exists m \; {\rm s.t.} \;
  c_{n} \leq \sigma_{m} \leq c_{0}
  \; \mathrm{or} \;
  \abs{\mu_{m}} > A_{0}
  \} \subset \Theta_{n} \ ,
  \nonumber
\end{eqnarray}
and let
\begin{eqnarray}
 \Gamma_{0}
  \equiv
  \{
  \theta \in \Theta
  \mid
  c_{0} \leq \sigma_{m} \leq B
  \; , \;
  \abs{\mu_{m}} \leq A_{0}
  ,\  (m=1,\ldots,M)\
  \}
  \; .
  \nonumber
\end{eqnarray}
Note that $\Theta_0 \subset \Gamma_0$, where $\Theta_0$ is the set of
true parameters.

\subsubsection{Partitioning the parameter space}
\label{sec:part-param-space}

In view of theorems in \cite{W1949}, \cite{R1981},
for the strong consistency of MLE on $\Theta_n$
under assumption~\ref{assumption:1}, \ref{assumption:2}, \ref{assumption:3} and \ref{assumption:4},
it suffices to prove that
$$
 \lim_{n \rightarrow \infty}
  \frac{
   \sup_{\theta \in \Gamma \cup \Theta_{n}'}
   \prod_{i=1}^{n} f(x_{i};\theta)
   }
   { \prod_{i=1}^{n} f(x_{i};\theta_{0}) } = 0, \quad a.e.
$$
for all closed $\Gamma\subset \Gamma_0$ not intersecting $\Theta_0$.
Note that for all $\Gamma$ and $\{x_{i}\}_{i=1}^{n}$,
\begin{eqnarray}
 {
  \sup_{\theta \in \Gamma \cup \Theta_{n}'}
  \prod_{i=1}^{n} f(x_{i};\theta)
  }
  =
  \max
  \left\{
  {
  \sup_{\theta \in \Gamma}
  \prod_{i=1}^{n} f(x_{i};\theta)
  }
  \; , \;
  {
  \sup_{\theta \in \Theta_{n}'}
  \prod_{i=1}^{n} f(x_{i};\theta)
  }
  \right\}
  \; . \;
  \nonumber
\end{eqnarray}
Furthermore
$$
   \lim_{n \rightarrow \infty}
  \frac{
   \sup_{\theta \in \Gamma}
   \prod_{i=1}^{n} f(x_{i};\theta)
   }
   { \prod_{i=1}^{n} f(x_{i};\theta_{0}) }
  = 0, \quad a.e.
$$
holds by
theorems in \cite{W1949}, \cite{R1981}.
Therefore it suffices to prove
\begin{eqnarray}
   \lim_{n \rightarrow \infty}
   \frac{
   \sup_{\theta \in \Theta_{n}'}
   \prod_{i=1}^{n} f(x_{i};\theta)
   }
   { \prod_{i=1}^{n} f(x_{i};\theta_{0}) }
   = 0, \quad a.e.
   \nonumber
\end{eqnarray}

Note that in the argument above the supremum of the likelihood
function over $\Gamma \cup \Theta_{n}'$ is considered separately for $\Gamma$
and $\Theta_n'$.  $\Gamma$ and $\Theta_n'$ form a covering of $\Gamma\cup
\Theta_n'$.  In our proof, we consider finer and finer finite coverings of
$\Theta_n'$.  As above, it suffices to prove that the ratio of the
supremum of the likelihood over each member of the covering to the
likelihood at $\theta_0$ converges to zero almost everywhere.

Let $\theta \in \Theta_{n}'$.
Let
$
 \mathscr{K}_{{\sigma} \leq c_{0}},
 \mathscr{K}_{{\sigma}\geq B},
 \mathscr{K}_{\abs{\mu} \geq A_{0}}
$
represent disjoint subsets of $\{1,\dots,M\}$ and
define
\begin{equation}
\scrk_{{R}} \equiv \{1, \dots, M\}
\backslash
\{\mathscr{K}_{{\sigma} \leq c_{0}} \cup \mathscr{K}_{{\sigma}\geq B} \cup \mathscr{K}_{\abs{\mu} \geq A_{0}}\}
 .
 \nonumber
\end{equation}
For any given
$
 \mathscr{K}_{{\sigma} \leq c_{0}},
 \mathscr{K}_{{\sigma}\geq B},
 \mathscr{K}_{\abs{\mu} \geq A_{0}}
$,
we define  a subset of $\Theta_n'$ by
\begin{eqnarray}
 \Theta_{n, \mathscr{K}}'
  & \equiv &
  \{
  \theta \in \Theta_{n}'
   \mid
  \sigma_{m} \leq c_{0},(m \in \mathscr{K}_{{\sigma} \leq c_{0}})
  \; ; \;
  \sigma_{m} \geq B,(m \in \mathscr{K}_{{\sigma}\geq B})
  \; ; \;
  \nonumber \\
 &&
  \qquad \qquad \quad
  c_{0} < \sigma_{m} < B,\; \abs{\mu_{m}}\geq A_{0},\; (m \in \mathscr{K}_{\abs{\mu} \geq A_{0}})
  \; ; \;
  \nonumber \\
 &&
  \qquad \qquad \quad
  c_{0} < \sigma_{m} < B,\; \abs{\mu_{m}}<A_{0},\; (m \in \scrk_{{R}})
  \}.
  \label{eq:def:Theta_n_scrk}
\end{eqnarray}
As above, it suffices to prove that for each choice of disjoint subsets
$
 \mathscr{K}_{{\sigma} \leq c_{0}},
 \mathscr{K}_{{\sigma}\geq B},
 \mathscr{K}_{\abs{\mu} \geq A_{0}},
$
\begin{eqnarray}
  \lim_{n \rightarrow \infty}
  \frac{
  \sup_{\theta \in \Theta_{n,\mathscr{K}}'}
  \prod_{i=1}^{n} f(x_{i};\theta)
  }
  { \prod_{i=1}^{n} f(x_{i};\theta_{0}) }
  = 0, \quad a.e.
  \nonumber
\end{eqnarray}
We fix
$
 \mathscr{K}_{{\sigma} \leq c_{0}},
 \mathscr{K}_{{\sigma}\geq B},
 \mathscr{K}_{\abs{\mu} \geq A_{0}},
$
from now on.

Next we consider coverings of $\bar{\Theta}_{\scrk_{{R}}}$.
Recall that $\bar{\Theta}_{\mathscr{K}}$, $f_{\mathscr{K}}(x;\theta_{\mathscr{K}})$ and $f_{\mathscr{K}}(x;\theta_{\mathscr{K}}, \rho)$
are defined in (\ref{eq:def:barTheta_scrk}), (\ref{eq:def:f_scrk}) and (\ref{eq:def:f_scrk_rho}).
The following lemma follows from lemma \ref{expectation-equality:lem} and compactness of $\bar{\Theta}_{\scrk_{{R}}}$.
\begin{lem}
\label{WaldThm1Type:lem}
Let $\mathscr{B}(\theta,\rho(\theta) )$ denote the open ball
with center $\theta$ and radius $\rho(\theta)$.
Then $\bar{\Theta}_{\scrk_{{R}}}$ can be covered by
a finite number of balls
$\mathscr{B}(\theta_{\scrk_{{R}}}^{(1)}, \rho(\theta_{\scrk_{{R}}}^{(1)}))
, \ldots, \mathscr{B}(\theta_{\scrk_{{R}}}^{(S)}, \rho(\theta_{\scrk_{{R}}}^{(S)}))
$
such that
\begin{eqnarray}
 E_{0}[\log{\{f_{\scrk_{{R}}}(x;\theta_{\scrk_{{R}}}^{(s)},\rho(\theta_{\scrk_{{R}}}^{(s)})) + \kappa_{0}\}}]
  + \lambda_{0}
  <
  E_{0}[\log{f(x;\theta_0)}]
  \; , \quad (s=1,\ldots,S) \; .
  \nonumber
\end{eqnarray}
\end{lem}
\noindent {\bf Proof: }\
{}From lemma \ref{expectation-equality:lem} we have
\begin{eqnarray}
 \lim_{\rho \rightarrow 0}
  E_{0}
  \left[
   \log{\{f_{\scrk_{{R}}}(x; \theta_{\scrk_{{R}}}, \rho) + \kappa_{0}\}}
  \right]
  =
  E_{0}
  \left[
   \log{\{f_{\scrk_{{R}}}(x; \theta_{\scrk_{{R}}}) + \kappa_{0}\}}
  \right].
  \nonumber
\end{eqnarray}
For each $\theta_{\scrk_{{R}}} \in \bar{\Theta}_{\scrk_{{R}}}$
\begin{eqnarray}
  E_{0}
  \left[
   \log{
   \left\{
    f_{\scrk_{{R}}}(x; \theta_{\scrk_{{R}}}) + \kappa_{0}
   \right\}
   }
  \right]
  +
  \lambda_{0}
  <
  E_{0}[\log{f(x;\theta_{0})}]
  \nonumber
\end{eqnarray}
holds.
Therefore for each
$\theta_{\scrk_{{R}}} \in \bar{\Theta}_{\scrk_{{R}}}$,
there exists a radius
$\rho(\theta_{\scrk_{{R}}}) > 0$ such that
\begin{eqnarray}
 E_{0}[\log{\{f_{\scrk_{{R}}}(x;\theta_{\scrk_{{R}}},\rho(\theta_{\scrk_{{R}}})) + \kappa_{0}\}}]
  + \lambda_{0}
  <
  E_{0}[\log{f(x;\theta_0)}].
  \nonumber
\end{eqnarray}
Since
\begin{eqnarray}
 \bar{\Theta}_{\scrk_{{R}}}
  \subset
  \bigcup_{\theta_{\scrk_{{R}}} \in \bar{\Theta}_{\scrk_{{R}}}}
  \mathscr{B}(\theta_{\scrk_{{R}}},\rho(\theta_{\scrk_{{R}}}))
  \nonumber
\end{eqnarray}
and the compactness of $\bar{\Theta}_{\scrk_{{R}}}$,
there exists a finite number of balls
$\mathscr{B}(\theta_{\scrk_{{R}}}^{(1)}, \rho(\theta_{\scrk_{{R}}}^{(1)})),\ldots,$
\\
$\mathscr{B}(\theta_{\scrk_{{R}}}^{(S)}, \rho(\theta_{\scrk_{{R}}}^{(S)}))$
which cover $\bar{\Theta}_{\scrk_{{R}}}$.
\qed

\bigskip
Based on lemma \ref{WaldThm1Type:lem} we partition $\Theta_{n,\mathscr{K}}'$.
Recall that we denote by $\theta_{\mathscr{K}}$ the subvector of $\theta \in \Theta$
consisting of the components in $\mathscr{K}$.
Define a subset of $\Theta_{n,\mathscr{K}}'$ by
\begin{eqnarray}
 \Theta_{n,\mathscr{K} ,s}'
  \equiv
  \{
  \theta \in \Theta_{n,\mathscr{K}}'
  \mid
  \theta_{\scrk_{{R}}} \in \mathscr{B}(\theta_{\scrk_{{R}}}^{(s)},\rho(\theta_{\scrk_{{R}}}^{(s)}))
  \}.
  \label{eq:def:Theta_scrk_n_s}
\end{eqnarray}
Then $\Theta_{n,\mathscr{K}}'$ is covered by
$\Theta_{n,\mathscr{K},1}', \ldots, \Theta_{n,\mathscr{K},S}'$ :
$$
\Theta_{n,\mathscr{K}}' = \bigcup_{s=1}^S \Theta_{n,\mathscr{K},s}' \   .
$$
Again it suffices to prove
that for each choice of
$
 \mathscr{K}_{{\sigma} \leq c_{0}},
 \mathscr{K}_{{\sigma}\geq B},
 \mathscr{K}_{\abs{\mu} \geq A_{0}},
 s
$
\begin{eqnarray}
   \lim_{n \rightarrow \infty}
   \frac{
   \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
   \prod_{i=1}^{n} f(x_{i};\theta)
   }
   { \prod_{i=1}^{n} f(x_{i};\theta_{0}) }
   = 0, \quad a.e.
  \label{goal3:LikelihoodRatio:eq}
\end{eqnarray}
We fix
$
 \mathscr{K}_{{\sigma} \leq c_{0}},
 \mathscr{K}_{{\sigma}\geq B},
 \mathscr{K}_{\abs{\mu} \geq A_{0}}$ and
$s$
from now on.
Because
$$
\lim_{n\rightarrow\infty} \frac{1}{n} \sum_{i=1}^{n}
\log{f(x_{i};\theta_0)} =
E_0[\log f(x;\theta_0)], \quad a.e.
$$
(\ref{goal3:LikelihoodRatio:eq}) is implied by
\begin{eqnarray}
\limsup_{n \rightarrow \infty} \frac{1}{n} \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
   \sum_{i=1}^{n} \log{f(x_{i};\theta)}
  <
   E_0[\log f(x;\theta_0)],
  \quad a.e.
  \label{goal:eq}
\end{eqnarray}
Therefore it suffices to prove (\ref{goal:eq}), which is a new
intermediate goal of our proof hereafter.

\subsubsection{Bounding the likelihood by four terms}
In this section we bound the likelihood function by four terms
depending on the positions of the observations $x_1, \dots, x_n$.
Let $R_{n}(V)$ denote the number of observations
which belong to a set $V \subset \mathbb{R}$.
\begin{lem}
 \label{lem:fourterms}
For $\theta \in \Theta_{n,\mathscr{K},s}'$
\begin{eqnarray}
 \frac{1}{n}\sum_{i=1}^{n}\log{f(x_{i};\theta)}
 & \leq &
  \frac{1}{n}\sum_{i=1}^{n}
  \log{
  \left\{
   f_{\scrk_{{R}}}(x_{i};\theta_{\scrk_{{R}}},\rho(\theta_{\scrk_{{R}}})) + 3{\kappa_{0}}
  \right\}
  }
  \nonumber \\
 & & +
  \frac{1}{n}R_{n}(\mathscr{A}_{0}) \cdot
  \log{
  \left(
   \frac{M v_0/c_0 + 2\kappa_{0}}{3\kappa_{0}}
  \right)
  }
  \nonumber \\
 & & +
  \frac{1}{n}R_n(J(\theta))\cdot
(
   -\log\kappa_{0}
)
  +
  \frac{1}{n}\sum_{x_{i} \in J(\theta)}
  \log{f(x_{i};\theta)}
  \; .
  \label{loglikelihood:lem:eq}
\end{eqnarray}
\end{lem}
\noindent {\bf Proof: }\
Let $\mathscr{K}_{{\sigma} > c_{0}}
= \{1,\dots,M\}\backslash\mathscr{K}_{{\sigma} \leq c_{0}}$
and
$\scrk_{c_{0}<\scparam <B} = \{1,\dots,M\}\backslash\{\mathscr{K}_{{\sigma} \leq c_{0}}\cup\mathscr{K}_{{\sigma} \geq B}\}$.
For $x \not\in J(\theta)$,
$f(x;\theta)\leq f_{\mathscr{K}_{{\sigma} > c_{0}}}
(x;\theta_{\mathscr{K}_{{\sigma} > c_{0}}})
+ \kappa_{0}$
holds.
Therefore
\begin{eqnarray}
 \frac{1}{n}\sum_{i=1}^{n}\log{f(x_{i};\theta)}
  & \leq & \frac{1}{n} \sum_{x_i \in J(\theta)} \log{f(x_{i};\theta)}
  + \frac{1}{n} \sum_{x_i \not \in J(\theta)}
  \log\left\{
       f_{\mathscr{K}_{{\sigma} > c_{0}}}(x;\theta_{\mathscr{K}_{{\sigma} > c_{0}}}) + {\kappa_{0}}
      \right\}
  \nonumber \\
 &= &
    \frac{1}{n}\sum_{i=1}^{n}
  \log{
  \left\{
   f_{\mathscr{K}_{{\sigma} > c_{0}}}(x;\theta_{\mathscr{K}_{{\sigma} > c_{0}}}) + {\kappa_{0}}
  \right\}
  }
  \nonumber \\
 && \qquad  +
  \frac{1}{n}\sum_{x_{i} \in J(\theta)}
  \left[
   \log
   f(x_i ; \theta)
   -
   \log
   \left\{
    f_{\mathscr{K}_{{\sigma} > c_{0}}}(x;\theta_{\mathscr{K}_{{\sigma} > c_{0}}}) + {\kappa_{0}}
   \right\}
  \right]
\label{eq:lemma9proof1}
\end{eqnarray}
Consider the second term on the right-hand side. We have
\begin{align*}
&  \frac{1}{n}\sum_{x_{i} \in J(\theta)}
  \left[
   \log
   f(x_i ; \theta)
   -
   \log
   \left\{
    f_{\mathscr{K}_{{\sigma} > c_{0}}}(x;\theta_{\mathscr{K}_{{\sigma} > c_{0}}}) + {\kappa_{0}}
   \right\}
  \right]\\
& \qquad\qquad \le
   \frac{1}{n}\sum_{x_{i} \in J(\theta)}
    \log
    f(x_i ; \theta)
   -
   \frac{1}{n}R_n(J(\theta)) \cdot \log{\kappa_{0}}
   \; .
\end{align*}
This takes care of the third and the fourth term of
  (\ref{loglikelihood:lem:eq}).

Now consider the first term on the right-hand side of
(\ref{eq:lemma9proof1}).
Note that
\begin{eqnarray}
    \frac{1}{n}\sum_{i=1}^{n}
  \log{
  \left\{
   f_{\mathscr{K}_{{\sigma} > c_{0}}}(x;\theta_{\mathscr{K}_{{\sigma} > c_{0}}}) + {\kappa_{0}}
  \right\}
  }
   &\leq &
  \frac{1}{n}\sum_{i=1}^{n}
  \log{
  \left\{
     f_{\scrk_{c_{0}<\scparam <B}}(x_{i};\theta_{\scrk_{c_{0}<\scparam <B}}) + 2{\kappa_{0}}
  \right\}
  }
  \nonumber
\end{eqnarray}
For $x\notin  \mathscr{A}_{0}$
$$
 f_{\mathscr{K}_{\abs{\mu} \geq A_{0}}}(x;\theta_{\mathscr{K}_{\abs{\mu} \geq A_{0}}})
  \leq
 {\kappa_{0}}.
$$
Therefore we obtain
\begin{eqnarray}
 \lefteqn{
 \frac{1}{n}\sum_{i=1}^{n}
  \log{
  \left\{
     f_{\scrk_{c_{0}<\scparam <B}}(x_{i};\theta_{\scrk_{c_{0}<\scparam <B}})
   + 2{\kappa_{0}}
  \right\}
  } }
  \nonumber \\
  & = &
  \frac{1}{n}\sum_{x_{i} \notin \mathscr{A}_{0}}
  \log{
  \left\{
   f_{\scrk_{c_{0}<\scparam <B}}(x_{i};\theta_{\scrk_{c_{0}<\scparam <B}})
   + 2{\kappa_{0}}
  \right\}
  }
  +
  \frac{1}{n}\sum_{x_{i} \in \mathscr{A}_{0}}
  \log{
  \left\{
   f_{\scrk_{c_{0}<\scparam <B}}(x_{i};\theta_{\scrk_{c_{0}<\scparam <B}})
   + 2{\kappa_{0}}
  \right\}
  }
  \nonumber \\
 & \leq &
    \frac{1}{n}\sum_{x_{i} \notin \mathscr{A}_{0}}
  \log{
  \left\{
   f_{\scrk_{{R}}}(x_{i};\theta_{\scrk_{{R}}})
   + 3{\kappa_{0}}
  \right\}
  }
  +
  \frac{1}{n}\sum_{x_{i} \in \mathscr{A}_{0}}
  \log{
  \left\{
   f_{\scrk_{c_{0}<\scparam <B}}(x_{i};\theta_{\scrk_{c_{0}<\scparam <B}})
   + 2{\kappa_{0}}
  \right\}
  }
  \nonumber \\
 & = &
  \frac{1}{n}\sum_{i=1}^{n}
  \log{
  \left\{
   f_{\scrk_{{R}}}(x_{i};\theta_{\scrk_{{R}}})
   + 3{\kappa_{0}}
  \right\}
  }
  \nonumber \\
 & & \qquad \qquad
  +
  \frac{1}{n}\sum_{x_{i} \in \mathscr{A}_{0}}
  \left[
   \log{
   \left\{
    f_{\scrk_{c_{0}<\scparam <B}}(x_{i};\theta_{\scrk_{c_{0}<\scparam <B}})
    + 2{\kappa_{0}}
   \right\}
   }
  -
   \log{
   \left\{
   f_{\scrk_{{R}}}(x_{i};\theta_{\scrk_{{R}}})
    + 3{\kappa_{0}}
   \right\}
   }
  \right]
  \nonumber  \\
  \label{eq:totyu}
\end{eqnarray}
Note that $f_{\scrk_{c_{0}<\scparam <B}}(x;\theta_{\scrk_{c_{0}<\scparam <B}}) \leq v_{0}/c_{0}$
from lemma~\ref{lem:BoundComponentByStepFunc}. Therefore
\begin{eqnarray}
 \lefteqn{
 \text{The r.h.s of (\ref{eq:totyu})}
 } & & \nonumber \\
 &\leq &
  \frac{1}{n}\sum_{i=1}^{n}
  \log{
  \left\{
   f_{\scrk_{{R}}}(x_{i};\theta_{\scrk_{{R}}})
   + 3{\kappa_{0}}
  \right\}
  }
  +
  \frac{1}{n}\sum_{x_{i} \in \mathscr{A}_{0}}
  \left[
   \log{
   \left\{
    v_0/c_0
    + 2{\kappa_{0}}
   \right\}
   }
   -
   \log{
   3\kappa_{0}
   }
  \right]
  \nonumber \\
 &\leq &
  \frac{1}{n}\sum_{i=1}^{n}
  \log{
  \left\{
     f_{\scrk_{{R}}}(x_{i};\theta_{\scrk_{{R}}},\rho(\theta_{\scrk_{{R}}})) + 3{\kappa_{0}}
  \right\}
  }
  +
  \frac{1}{n} R_{n}(\mathscr{A}_{0}) \cdot
  \log{
  \left(
   \frac{v_0/c_0 + 2{\kappa_{0}}}{3{\kappa_{0}}}
  \right)
  }.
  \nonumber
\end{eqnarray}
This takes care of the first and the second term of
(\ref{loglikelihood:lem:eq}).
\qed

\bigskip
From lemma~\ref{WaldThm1Type:lem} and the strong law of large
numbers the first term on the right hand side of (\ref{loglikelihood:lem:eq})
converges to the expectation of a density which has less than $M$ components
and the expectation is less than that of the true density by theorem \ref{kappa-lambda:thm}.
The second term
converges to a small value because the relative frequency on $\mathscr{A}_{0}$
is very small.
The third term
also converges to small value because the relative frequency on $J(\theta)$
is very small.
The fourth term
is somewhat complicated.
The component in $\mathscr{K}_{\sigma \le c_{0}}$ may have high peaks.
However the widths of the peaks are very narrow and the relative frequency on the interval is very small.
Hence the fourth term makes little contribution to the likelihood.
Therefore the mean log likelihood (the left hand side of (\ref{loglikelihood:lem:eq})) converges to
a value which is less than that of the true density.
In the following we consider the details.

The first term and the second term are easy.

\bigskip
\noindent
{\bf The first term}: By lemma~\ref{WaldThm1Type:lem} and the strong law of large
numbers we have
\begin{eqnarray}
\label{bounding-barf:eq}
\lim_{n\rightarrow\infty}  \frac{1}{n}\sum_{i=1}^{n}
  \log{
  \left\{
     f_{\scrk_{{R}}}(x_{i};\theta_{\scrk_{{R}}},\rho(\theta_{\scrk_{{R}}})) + 4{\kappa_{0}}
  \right\}
  }
 <
 E_0[\log f(x;\theta_0)] - 4\lambda_{0} , \quad a.e.
\end{eqnarray}

\bigskip
\noindent
{\bf The second term}:
By (\ref{A-zero:condition:eq}) and the strong law of large
numbers we have
\begin{eqnarray}
 \label{bounding-A-zero-term:eq}
  \lim_{n\rightarrow\infty}
  \frac{1}{n}R_{n}(\mathscr{A}_{0}) \cdot
  \log{
  \left(
   \frac{v_0/c_0 + 2\kappa_{0}}{3\kappa_{0}}
  \right)
  }
  <
  {\lambda_{0}}
  , \quad a.e.
\end{eqnarray}

Note that we have $-4\lambda_0$ from the first term and $\lambda_0$
from the second term. In the rest of our proof we show that
both the third term and the fourth term can be bounded by $\lambda_0$.

\subsubsection{Bounding the third term}

The third term can be bounded by dividing the interval $[-A_n, A_n]$
into short intervals of length $2\nu(c_0)$.
\begin{lem}
\label{boundedRJ:lem}
 \begin{eqnarray}
 \limsup_{n \rightarrow \infty}
  \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
   \frac{1}{n} R_n(J(\theta))
   \leq 3M \cdot u_{0} \cdot 2\nu(c_{0}),
  \quad a.e.
  \nonumber
\end{eqnarray}
\end{lem}
\noindent {\bf Proof: }\
\begin{figure}[htbp]
 \begin{center}
  \includegraphics{i.eps}
  \caption{Division of $J_{0}^{(n)}$ by
  short intervals of length $2\nu(c_{0})$.}
  \label{divided_J0n:tpic:fig}
 \end{center}
\end{figure}
Let $\epsilon > 0$ be arbitrarily fixed
and let $J_{0}^{(n)} \equiv [-A_{n}, A_{n}]$.
We divide $J_{0}^{(n)}$ from $-A_{n}$ to $A_{n}$ by short intervals of
length $2\nu(c_{0})$.  In right end of the intervals of $J_{0}^{(n)}$,
overlap of two short intervals of length $2\nu(c_{0})$ is allowed and the
right end of a short interval coincides with the right end of $J_0^{(n)}$.
See Figure \ref{divided_J0n:tpic:fig}.
Let $k_{n}({c_{0}})$ be the number of
short intervals and let
$I_{1}^{(n)}(c_{0}),\ldots,I_{k_{n}(c_{0})}^{(n)}(c_{0})$ be the divided short
intervals.
Then we have
\begin{eqnarray}
 k_{n}(c_{0})
  \leq
   \frac{2A_{n}}{2\nu(c_{0})} + 1
   =
   \frac{A_{n}}{\nu(c_{0})} + 1
   =
   \frac{A_{0} \cdot n^{\frac{2 + \zeta}{\beta - 1}}}{\nu(c_{0})} + 1
  \; .
  \label{bounding-number-of-short-intervals-c-zero:eq}
\end{eqnarray}
Note that any interval in $J_{0}^{(n)}$ of length $2\nu(c_{0})$ is
covered by at most $3$ small intervals
from $\{I_{1}^{(n)}(c_{0}),\ldots,I_{k_{n}(c_{0})}^{(n)}(c_{0})\}$.
Now consider
$J(\theta) = \bigcup_{m=1}^{K}
[\mu_{m}-\nu({\sigma}_{m}),\mu_{m}+\nu({\sigma}_{m}))$.
Since
$[\mu_{m}-\nu({\sigma}_{m}),\mu_{m}+\nu({\sigma}_{m}))
\; , \; (m=1,\ldots,K)$ ,
are intervals of length less
than or equal to  $2\nu(c_{0})$,
$J(\theta)$ is covered by at most $3M$ short intervals.
Then
\begin{eqnarray}
 \lefteqn{
  \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
  \frac{1}{n} R_n(J(\theta))
  - 3M \cdot u_{0} \cdot 2\nu(c_{0})
  > \epsilon
  } & &
  \nonumber \\
 & \Rightarrow &
  \{x_{n,1} < - A_{n} \quad \mathrm{or} \quad x_{n,n} > A_{n}\}
  \nonumber \\
  & & \mathrm{or}
  \nonumber \\
 & &
  \{{1} \leq \exists k \leq {k_{n}(c_{0})} \; , \;
  \frac{1}{n} R_n(I_{k}(c_{0}))
  - u_{0} \cdot 2\nu(c_{0})
  > \frac{\epsilon}{3M}
  \}
  \; .
  \label{supR_InequalityUsingSmallIntervals:eq}
\end{eqnarray}
By lemma \ref{expanding-A:lem},
$\sum_n \mathrm{Prob}(x_{n,1} < - A_{n} \  \mathrm{or} \  x_{n,n} > A_{n}) <
\infty$ and the first event on  the right-hand side of
(\ref{supR_InequalityUsingSmallIntervals:eq}) can be ignored.
We only need to consider the second event.
We will use the same logic in the proofs of  lemmas
 \ref{99-2:goal:lem} and  \ref{99-1:goal:lem} below.
Then
\begin{eqnarray}
 & &
 \mathrm{Prob}
  \left(
   \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
  \frac{1}{n} R_n(J(\theta))
  - 3M \cdot u_{0} \cdot 2\nu(c_{0})
  > \epsilon
  \right)
 \nonumber \\
 & & \hspace{6cm} \leq
  \sum_{k=1}^{k_{n}(c_{0})}
  \mathrm{Prob}
  \left(
   \frac{1}{n} R_n(I_{k}(c_{0}))
   - u_{0} \cdot 2\nu(c_{0})
   > \frac{\epsilon}{3M}
  \right)
  \; .
  \nonumber
\end{eqnarray}
Recall that, for any set $V \subset \mathbb{R}$,
we denote by $P_0(V)$ the probability of
$V$ under the true density in (\ref{eq:def:P_0})
and denote by $R_{n}$ the number of observations
which belong to $V$ as in lemma~\ref{lem:fourterms}.
Since
\begin{eqnarray}
 P_0(I_{k}(c_{0})) \leq u_{0} \cdot 2\nu(c_{0}) ,
  \quad
  (k=1,\ldots,k_{n}(\theta)),
  \nonumber
\end{eqnarray}
$R_n(V) \sim \mathrm{Bin}(n , P_0(V))$
and from Okamoto's inequality (\cite{O1958}),
we obtain \begin{eqnarray}
 \lefteqn{
  \mathrm{Prob}
  \left(
   \frac{1}{n} R_n(I_{k}(c_{0}))
   - u_{0} \cdot 2\nu(c_{0})
   > \frac{\epsilon}{3M}
  \right)
  } & & \nonumber \\
 & \leq &
  \mathrm{Prob}
  \left(
   \frac{1}{n} R_n(I_{k}(c_{0}))
   - P_0(I_{k}(c_{0}))
   > \frac{\epsilon}{3M}
  \right)
  \nonumber \\
 & \leq &
  \exp{
  \left(
   -\frac{2n\epsilon^2}{9M^{2}}
  \right)} .
  \nonumber
\end{eqnarray}
Therefore from (\ref{bounding-number-of-short-intervals-c-zero:eq})
\begin{eqnarray}
 \lefteqn{
 \mathrm{Prob}
  \left(
   \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
  \frac{1}{n} R_n(J(\theta))
  - 3M \cdot u_{0} \cdot 2\nu(c_{0})
  > \epsilon
  \right)
  } & & \nonumber \\
 & &  \hspace{4cm}
  \le
  \left(
   \frac{A_{0} \cdot n^{\frac{2 + \zeta}{\beta - 1}}}{\nu(c_{0})} + 1
  \right)
  \cdot
  \exp{
  \left(
   -\frac{2n\epsilon^2}{9M^{2}}
  \right)}.
  \nonumber
\end{eqnarray}
When we sum this over $n$,
the resulting series on the right converges.
Hence by the Borel-Cantelli lemma, we have
\begin{eqnarray}
 \mathrm{Prob}
  \left(
    \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
    \frac{1}{n} R_n(J(\theta))
    - 3M \cdot u_{0} \cdot 2\nu(c_{0})
   > \epsilon
   \quad i.o.
  \right) = 0.
  \nonumber
\end{eqnarray}
Because $\epsilon > 0$ was arbitrary,
we obtain
\begin{eqnarray}
 \limsup_{n \rightarrow \infty}
 \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
  \frac{1}{n} R_n(J(\theta))
  \leq
  3M \cdot u_{0} \cdot 2\nu(c_{0}),
  \quad a.e.
  \nonumber
\end{eqnarray}
\qed

\bigskip
By this lemma
and
(\ref{eq:condition:proof:mainthm})
we have
\begin{eqnarray}
  \limsup_{n \rightarrow \infty}
  \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
  \frac{1}{n} R_n(J(\theta)) \cdot
  \left(-\log{{\kappa_{0}}}\right)
  \leq 3M \cdot u_{0} \cdot 2\nu(c_{0}) \cdot
  \abslr{\log{{\kappa_{0}}}}
  < {\lambda_{0}}
  \quad a.e.
 \label{boundingRJ:eq}
\end{eqnarray}
This bounds the third term on the right-hand side of
(\ref{loglikelihood:lem:eq}) from above.

\subsubsection{Bonding the fourth term}

Finally
we bound the fourth term on the right-hand side of
(\ref{loglikelihood:lem:eq}) from above.
From lemma~\ref{lem:BoundMixtureByStepFunc} we have
\begin{equation}
 \mathrm{1}_{J(\theta)}(x) \cdot
 \sum_{m=1}^{M}
 \alpha_{m} f_{m}(x; \mu_{m}, \sigma_{m})
 \le
 \sum_{t=1}^{T(\theta)}H(J_{t}(\theta)) \cdot
 \mathrm{1}_{J_{t}(\theta)}(x)
 \quad , \quad
 (x \in J(\theta))
 \label{bounded-by-uniform-density-2:main-thm:eq}
\end{equation}

We now classify the intervals $J_{t}(\theta),\ t=1,\ldots, T(\theta),$
by the height $H(J_t(\theta))$.
Let
\begin{eqnarray}
 c_{n}' \equiv c_{0} \cdot \exp{(-n^{1/4})}
  \label{eq:def:c_n_pr}
\end{eqnarray}
and define $\tau_n(\theta)$ and $\tau_n'(\theta)$ by
\begin{eqnarray}
 \tau_n(\theta) \equiv
 \{
 t \in \{1,\ldots,T(\theta)\}
 \mid
 H(J_{t}(\theta)) \leq M v_0/c_{n}'
 \}
\ , \
 \tau_n'(\theta) \equiv
 \{1,\ldots,T(\theta)\}\backslash \tau_n(\theta)
 \ .
 \label{eq:def:tau_n_and_tau_n_pr}
\end{eqnarray}
See Figure \ref{fig:CnCnd}.
\begin{figure}[htbp]
 \begin{center}
  \includegraphics{CnCnd.eps}
  \caption{Example of classification of the intervals
  by the height ($\tau_{n}(\theta) = \{1,4\} \; , \; \tau_{n}'(\theta) = \{2,3\} $).}
  \label{fig:CnCnd}
 \end{center}
\end{figure}

Now suppose that the following inequality holds.
\begin{eqnarray}
 \lefteqn{
  \limsup_{n \rightarrow \infty}
  \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
  \left[
   \sum_{t=1}^{T(\theta)}
   \frac{1}{n}
   R_{n}(J_{t}(\theta))
   \log{ H(J_{t}(\theta)) }
   \right.
  } & & \nonumber \\
 & &
  - \left.
   3\left\{
     \sum_{t \in \tau_n(\theta)}
     u_{0} \cdot {\xi(H(J_{t}(\theta)))} \cdot
     \log{H(J_{t}(\theta))}
     +
     \sum_{t \in \tau_n'(\theta)}
     \frac{2}{n}\log{H(J_{t}(\theta))}
    \right\}
  \right]
  \leq 0 ,
  \quad a.e.
  \nonumber \\
  \label{2:goal:eq}
\end{eqnarray}
From
(\ref{eq:condition:proof:mainthm}),
and noting that $\log y/y^{\tilde{\beta}}$ is
decreasing in $y^{\tilde{\beta}} \ge e$,
we have
\begin{eqnarray}
 3 \cdot \sum_{t \in \tau_n(\theta)}
     {u_{0}} \cdot {\xi(H(J_{t}(\theta)))} \cdot
     \log{H(J_{t}(\theta))}
   & \leq &
    3 \cdot 2M \cdot u_{0} \cdot
    \xi(v_0/c_0)
    \cdot \log(v_0/c_{0})
   <
  {\lambda_{0}} ,
  \nonumber \\
 3 \cdot \sum_{t \in \tau_n'(\theta)}
  \frac{2}{n}\log{H(J_{t}(\theta))}
  &\leq&
  3 \cdot 2{M} \cdot \frac{2}{n} \cdot
  \log\frac{Mv_0}{c_n}
  \rightarrow 0 .
  \label{1:limit:ineq:eq}
\end{eqnarray}
Then from
(\ref{bounded-by-uniform-density-2:main-thm:eq}),
(\ref{2:goal:eq})
and
(\ref{1:limit:ineq:eq}),
the fourth term on the right-hand side of
(\ref{loglikelihood:lem:eq}) is bounded from above as
\begin{eqnarray}
\label{eq:bound-4nd-term:eq}
\limsup_{n\rightarrow\infty}\frac{1}{n} \sup_{\theta\in
  \Theta_{n,\mathscr{K},s}'}
\sum_{x_i \in J(\theta)} \log
f(x_i; \theta) \le {\lambda_{0}}
\quad a.e.
\end{eqnarray}
Combining
(\ref{loglikelihood:lem:eq}),
(\ref{bounding-barf:eq}),
(\ref{bounding-A-zero-term:eq}),
(\ref{boundingRJ:eq}) and
(\ref{eq:bound-4nd-term:eq}) we obtain
\begin{eqnarray}
  \limsup_{n \rightarrow \infty}
  \sup_{\theta \in \Theta_{n,\mathscr{K},s}'}
  \frac{1}{n}  \sum_{i=1}^{n}
  \log{f(x_i; \theta)}
 & \leq &
E_{0}[\log{f(x;\theta_0)}]
   - {\lambda_{0}} ,
   \quad a.e.
   \nonumber
\end{eqnarray}
and ~(\ref{goal:eq}) is satisfied.  Therefore it suffices to
prove (\ref{2:goal:eq}), which is a new goal of our proof.

We now consider further finite covering of $\Theta_{n,\mathscr{K},s}'$.  For
any $T \; (1 \leq T \leq 2M)$ and $\tau \subset
\{1,\dots,T(\theta)\}$, define a subset of
$\Theta_{n,\mathscr{K},s}$ by
\begin{eqnarray}
 \Theta_{n,\mathscr{K},s,T,\tau}'
  \equiv
  \{
  \theta \in \Theta_{n,\mathscr{K},s}'
  \mid
  T(\theta) = T
  \; ,\;
  \tau_{n}(\theta) = \tau
  \}
 \; .
 \label{eq:def:Theta_scrk_s_T_tau}
\end{eqnarray}
Then (\ref{2:goal:eq}) is derived from the following two lemmas.
\begin{lem}
 \begin{eqnarray}
  \limsup_{n \rightarrow \infty}
   \sup_{\theta \in \Theta_{n,\mathscr{K},s,T,\tau}'}
   \left[
    \sum_{t \in \tau'}
    \frac{1}{n}
    R_{n}(J_{t}(\theta)) \cdot
    \log{ H(J_{t}(\theta)) }
   -
   3 \sum_{t \in \tau'}
   \frac{2}{n}\log{H(J_{t}(\theta))}
  \right]
  \leq 0
  \quad a.e. \ ,
  \nonumber \\
  \nonumber
 \end{eqnarray}
where $\tau' = \{1,\dots,T\}\backslash \tau$.
\label{99-2:goal:lem}
\end{lem}
\noindent {\bf Proof: }\
Let $\delta > 0$ be  any fixed positive real constant and
let $\mu_{t}'(\theta)$ denote the middle point of $J_{t}(\theta)$.
Here, we consider the probability of the event that
\begin{eqnarray}
 \sup_{\theta \in \Theta_{n,\mathscr{K},s,T,\tau}'}
  \left[
   \sum_{t \in \tau'}
   \frac{1}{n}
   R_{n}(J_{t}(\theta)) \cdot
   \log{ H(J_{t}(\theta)) }
   -
   3 \sum_{t \in \tau'}
   \frac{2}{n}\log{H(J_{t}(\theta))}
  \right]
  > 2M\delta .
  \label{99-2:RareEvent:eq}
\end{eqnarray}

Since
$H(J_{t}(\theta)) > M v_0/c_{n}'$
holds for $t \in \tau'$,
we obtain by lemma \ref{lem:BoundWidthOfInterval}
\begin{eqnarray}
 W(J_{t}(\theta))
 & \leq &
  v_{2} \cdot
  \left(
   \frac{c_{n}'}{M v_0}
  \right)
  ^{\tilde{\beta}}
  =
  v_{2} \cdot
  \left(
   \frac{c_{0}}{M v_{0}}
  \right)
  ^{\tilde{\beta}}
  \cdot \exp{(- {\tilde{\beta}} \cdot n^{1/4})}.
  \nonumber
\end{eqnarray}
Let
\begin{eqnarray}
 v_{3} & \equiv &
  v_{2} \cdot
  \left(
   \frac{c_{0}}{M v_{0}}
  \right)
  ^{\tilde{\beta}},
  \nonumber \\
 w_{n} & \equiv & \frac{v_{3}}{2} \cdot \exp{(- {\tilde{\beta}} \cdot n^{1/4})},
  \label{eq:def:w_n}
  \\
  R_{n}[\mu, w] & \equiv & R_{n}([\mu - w, \mu + w]).
   \label{eq:def:R_n-mu-w}
\end{eqnarray}
Noting that for $t \in \tau'$, the length of $J_t(\theta)$
is less than or equal to $2w_{n}$,
the following relation holds.
\begin{eqnarray}
 \lefteqn{
  \textrm{The event (\ref{99-2:RareEvent:eq}) occurs. }
  } & &
  \nonumber \\
 & \Rightarrow &
 \sup_{\theta \in \Theta_{n,\mathscr{K},s,T,\tau}'}
  \left[
   \sum_{t\in \tau'}
    \left(
     \frac{1}{n}
      R_n[\mu_{t}'(\theta), w_{n}]
     - 3 \cdot \frac{2}{n}
    \right)
    \cdot
   \log \frac{M v_0}{c_{n}}
  \right]
 > 2M\delta
 \nonumber \\
 & \Rightarrow &
  \exists \theta \in \Theta_{n,\mathscr{K},s,T,\tau}' ,\;
  \exists t \in \tau'
  \quad \text{s.t.} \quad
   \left(
    \frac{1}{n}
    R_n[\mu_{t}'(\theta), w_{n}]
    - 3 \cdot \frac{2}{n}
   \right)
   \cdot
  \log\frac{Mv_0}{c_{n}}
  > \delta
  \nonumber \\
 & \Rightarrow &
  \exists \theta \in \Theta_{n,\mathscr{K},s,T,\tau}' ,\;
  \exists t \in \tau'
  \quad \text{s.t.} \quad
  R_n[\mu_{t}'(\theta), w_{n}]
  \geq {6}
  \nonumber \\
 & \Rightarrow &
  \sup_{-\infty < {\mu'} < \infty} R_n[{\mu'}, w_{n}] \geq 6
   \; .
  \label{99-2:rarer-event:eq}
\end{eqnarray}
Below, we consider the probability of the event
that (\ref{99-2:rarer-event:eq}) occurs.
We divide $J_{0}^{(n)} = [-A_{n}, A_{n}]$
 from $-A_{n}$ to $A_{n}$
by short intervals of length $2w_{n}$ as
in the proof of lemma \ref{boundedRJ:lem}.
Let $k({w_{n}})$ be the number of
short intervals and let
$I_{1}(w_{n}),\ldots,I_{k(w_{n})}(w_{n})$ be the divided short
intervals.
Then we have
\begin{eqnarray}
 k(c_{n}') \leq
   \frac{2A_{n}}{2w_{n}} + 1
   =
   \frac{A_{0} \cdot n^{\frac{2 + \zeta}{\beta - 1}}}{\nu(c_{0})} + 1
   \; .
  \label{99-2:bound-k_c_n:eq}
\end{eqnarray}
Since any interval in $J_{0}$ of length $2w_{n}$ is
covered by at most $3$ small intervals
from $I_{1}(w_{n}),\ldots,I_{k(w_{n})}(w_{n})$
and from lemma \ref{expanding-A:lem},
\begin{eqnarray}
 \sup_{-\infty < {\mu'} < \infty} R_n[{\mu'}, w_{n}] \geq 6
  & \Rightarrow &
  1 \leq \exists k \leq k(w_{n}) \; ,\;
   R_n(I_{k}(w_{n})) \geq 2
  \; .
  \nonumber
\end{eqnarray}
Note that $R_n(I_{k}(w_{n}))
\sim \mathrm{Bin}(n , P_0(I_{k}(w_{n})))$
and $P_0(I_{k}(w_{n})) \le 2w_{n} u_{0}$.
Therefore from (\ref{99-2:bound-k_c_n:eq})
we have
\begin{eqnarray}
 \lefteqn{
  \sum_{k=1}^{k(w_{n})}
  \mathrm{Prob}
  \left(
   R_n(I_{k}(w_{n})) \geq 2
  \right)
  \leq
  \left(
   \frac{A_{n}}{w_{n}} + 1
  \right) \cdot
  \left\{
  \max_{1\le k \le k(w_{n})}
  \mathrm{Prob}(R_n(I_{k}(w_{n})) \geq 2)
  \right\}
  } & &
   \nonumber \\
 &  &
  \leq
  \left(
   \frac{A_{n}}{w_{n}} + 1
  \right)
  \sum_{k=2}^{n}
  \begin{pmatrix} n \\ k \end{pmatrix}
  (2w_{n} u_{0})^{k}(1 - 2w_{n} u_{0})^{n-k}
   \nonumber \\
 &  &
  \leq
  \left(
   \frac{A_{n}}{w_{n}} + 1
  \right)
  \sum_{k=2}^{n}\frac{n^{k}}{k!}(2w_{n} u_{0})^{k}
  \leq
   \left(
   \frac{A_{n}}{w_{n}} + 1
  \right)
  (2nw_{n} u_{0})^{2}
  \sum_{k=0}^{n}\frac{1}{k!}(2nw_{n} u_{0})^{k}
  \nonumber \\
 &  &
  \leq
    \left(
   \frac{A_{n}}{w_{n}} + 1
  \right)
  (2nw_{n} u_{0})^{2}
  \exp{(2nw_{n} u_{0})}
  \; .
  \nonumber
\end{eqnarray}
When we sum this over $n$,
resulting series on the right converges.
Hence by the Borel-Cantelli lemma and the fact that $\delta > 0$ was arbitrary,
we obtain
\begin{eqnarray}
  \limsup_{n \rightarrow \infty}
   \sup_{\theta \in \Theta_{n,\mathscr{K},s,T,\tau}'}
   \left[
    \sum_{t \in \tau'}
    \frac{1}{n}
    R_{n}(J_{t}(\theta)) \cdot
    \log{ H(J_{t}(\theta)) }
    -
    3 \sum_{t \in \tau'}
    \frac{2}{n}\log{H(J_{t}(\theta))}
  \right]
   \leq 0
   \quad a.e.
   \nonumber
 \end{eqnarray}
\qed

\begin{lem}
 \begin{eqnarray}
  \lefteqn{
   \limsup_{n \rightarrow \infty}
   \sup_{\theta \in \Theta_{n,\mathscr{K},s,T,\tau}'}
   \left[
    \sum_{t \in \tau}
    \frac{1}{n}
    R_{n}(J_{t}(\theta)) \cdot
    \log{ H(J_{t}(\theta)) }
   \right.
   }& &
   \nonumber \\
  & & \hspace{4cm}\left.
   -
   3 \sum_{t \in \tau}
   u_{0} \cdot \xi(H(J_{t}(\theta)))
   \cdot \log{H(J_{t}(\theta))}
  \right]
  \leq 0
  \quad a.e.
  \nonumber
 \end{eqnarray}
 \label{99-1:goal:lem}
\end{lem}
\noindent {\bf Proof: }\
Let $\delta > 0$ be any fixed positive real constant
and let
\begin{eqnarray}
 h_{n}
  \equiv
  \frac{\delta}{12}
  \left\{
   u_{0} \cdot
   \log{
     \frac{M v_0}{c_{n}'}
   }
  \right\}^{-1}
 \; .
 \label{99-1:h_n:def:eq}
\end{eqnarray}
Since
$v_0/c_0 \leq H(J_{t}(\theta)) \leq M v_0/c_{n}'$,
we have
$\xi(M v_0/c_{n})\leq \xi(H(J_{t}(\theta))) \leq \xi(v_0/c_{0})$.
We divide the interval
$[\xi(M v_0/c_{n}'), \xi(v_0/c_{0})]$
from
$\xi(c_0/v_0)$ to $\xi(M v_0/c_{n}')$
by short intervals of length $h_{n}$.
In the left end
$\xi(M v_0/c_{n}')$
of the interval
$[\xi(M v_0/c_{n}'), \xi(v_0/c_{0})]$,
overlap of two short intervals of length $h_{n}$ is allowed and the
left  end of a short interval is equal to
$\xi(M v_0/c_{n}')$.
Let $l_{n}$ be the number of
short intervals of length $h_{n}$ and define $w_{l}^{(n)}$ by
\begin{eqnarray}
 2w_{l}^{(n)} \equiv
 \begin{cases}
   \xi(v_0/c_{0}) - (l - 1)h_{n}, & 1 \leq l \leq l_{n},\\
    \xi(M v_0/c_{n}'), & l = l_{n} + 1.
 \end{cases}
 \label{eq:def:w_l-n}
\end{eqnarray}
Then we have
\begin{eqnarray}
 l_{n} \leq
   \frac{\xi(v_0/c_{0})}{h_{n}} + 1
  \; .
  \label{99-1:l_n:ineq:eq}
\end{eqnarray}
Let
\begin{eqnarray}
 \psi(y) \equiv \xi^{-1}(y) = \left(\frac{v_{2}}{y}\right)^{1/\tilde{\beta}} \ ,
  \qquad (y > 0) \ ,
  \nonumber
\end{eqnarray}
where $\xi^{-1}(\cdot)$ is the inverse function of $\xi(\cdot)$.
Next we consider the probability of the event that
\begin{eqnarray}
 \lefteqn{
  \sup_{\theta \in \Theta_{n,\mathscr{K},s,T,\tau}'}
  \left[
   \sum_{t \in \tau}
   \frac{1}{n}
   R_{n}(J_{t}(\theta)) \cdot
   \log{ H(J_{t}(\theta)) }
  \right.
   }& &
   \nonumber \\
 & & \hspace{4cm}
  \left.
   -
   3 \sum_{t \in \tau}
   u_{0} \cdot \xi(H(J_{t}(\theta)))
   \cdot \log{H(J_{t}(\theta))}
  \right]
  > 2M\delta .
  \label{99-1:rare-event:eq}
\end{eqnarray}
For this event the following relation holds.
\begin{eqnarray}
 \lefteqn{
  \textrm{The event (\ref{99-1:rare-event:eq}) occurs. }
  } & &
  \nonumber \\
 & \Rightarrow &
 \exists \theta \in \Theta_{n,\mathscr{K},s,T,\tau}'
 , \;
  \forall t \in \tau ,\;
  1 \leq \exists l(t) \leq l_{n}
  \quad {\rm s.t.} \;
  \nonumber \\
 & &
 \psi(2w_{l(t)}^{(n)}) \leq H(J_{t}(\theta))
 \leq \psi(2w_{l(t)+1}^{(n)})
 \quad {\rm and} \;
 \nonumber \\
 & &
 \sum_{t \in \tau}
   \left(
    \frac{1}{n}
    R_n[\mu_{t}'(\theta), w_{l(t)}^{(n)}]
   -
   3u_{0} \cdot 2w_{l(t)+1}^{(n)}
  \right)
  \cdot
 \log\psi(2w_{l(t)+1}^{(n)})
 > 2M\delta
 \nonumber
 \\
 & \Rightarrow &
  \exists \theta \in \Theta_{n,\mathscr{K},s,T,\tau}'
  \; , \;
  \exists t \in \tau
  , \;
  1 \leq
  \exists l(t)
  \leq l_{n}
  \quad {\rm s.t.} \;
  \nonumber \\
 & &
  \psi(2w_{l(t)}^{(n)}) \leq H(J_{t}(\theta)) \leq \psi(2w_{l(t)+1}^{(n)})
  \quad {\rm and} \;
  \nonumber \\
 & &
   \left(
    \frac{1}{n}
    R_n[\mu_{t}'(\theta), w_{l(t)}^{(n)}]
      -
    3u_{0} \cdot 2w_{l(t)+1}^{(n)}
   \right)
   \cdot
  \log\psi(2w_{l(t)+1}^{(n)})
  > \delta
  \nonumber \\
   & \Rightarrow &
  1 \leq
  \exists l
  \leq l_{n}
  \quad {\rm s.t.} \;
  \nonumber \\
 & &
   \sup_{-\infty < {\mu'} < \infty}
   \left(
    \frac{1}{n}
    R_n[{\mu'}, w_{l}^{(n)}]
      -
    3u_{0} \cdot 2w_{l+1}^{(n)}
   \right)
   \cdot
  \log\psi(2w_{l+1}^{(n)})
  > \delta
  \nonumber \\
 & \Rightarrow &
  1 \leq \exists l \leq l_{n}
  \quad {\rm s.t.} \;
  \nonumber \\
 & &
  \sup_{-\infty < {\mu'} < \infty}
  \left\{
  \left(
   \frac{1}{n} R_n[{\mu'}, w_{l}^{(n)}]
   -
   3u_{0} \cdot 2w_{l}^{(n)}
   \right) \cdot
  \log\psi(2w_{l+1}^{(n)})
  \right.
  \nonumber \\
 & & \qquad \qquad \qquad \qquad \quad
  +
  \left.
   3u_{0} \cdot (2w_{l}^{(n)} - 2w_{l+1}^{(n)})
   \cdot \log \psi(2w_{l+1}^{(n)})
  \right\}
  > \delta
 \label{99-1:rarer-event:eq}
\end{eqnarray}
Then from (\ref{99-1:h_n:def:eq}) and lemma \ref{expanding-A:lem}
the following relation holds.
\begin{eqnarray}
 \lefteqn{
  \textrm{The event (\ref{99-1:rarer-event:eq}) occurs.}
  } & &
 \nonumber \\
 & \Rightarrow &
  {1}\leq \exists l \leq {l_{n}}
  \quad {\rm s.t.} \;
  \nonumber \\
 & &
   \sup_{-\infty < {\mu'} < \infty}
   \frac{1}{n}
   \left(
    R_n[{\mu'}, w_{l}^{(n)}]
    -
    3u_{0} \cdot 2w_{l}^{(n)}
   \right) \cdot
   \log \psi(2w_{l+1}^{(n)})
   > \frac{\delta}{2}
  \nonumber \\
  \nonumber \\
 & \Rightarrow &
  {1}\leq \exists l \leq {l_{n}}
  \quad {\rm s.t.} \;
  \nonumber \\
 & &
  \sup_{-A_{n} < {\mu'} < A_{n}}
  \frac{1}{n}
   \left(
    R_n[{\mu'}, w_{l}^{(n)}]
    -
    3u_{0} \cdot 2w_{l}^{(n)}
   \right) \cdot
   \log \psi(2w_{l+1}^{(n)})
   > \frac{\delta}{2}
  \label{99-1:rarest-event:eq}
\end{eqnarray}
Below, we consider the probability of the event
that
(\ref{99-1:rarest-event:eq}) occurs.
We divide $J_{0}^{(n)}$ from $-A_{n}$ to $A_{n}$
by short intervals of length $2w_{l}^{(n)}$ as
in the proof of lemma \ref{boundedRJ:lem}.
Let $k({w_{l}^{(n)}})$ be the number of
short intervals and let
$I_{1}(w_{l}^{(n)}),\ldots,I_{k(w_{l}^{(n)})}(w_{l}^{(n)})$ be the divided short
intervals.
Then we have
\begin{eqnarray}
 k(w_l^{(n)}) \leq
   \frac{2A_{n}}{2w_{l}^{(n)}} + 1
  \; .
  \label{99-1:bound-k_c_n:eq}
\end{eqnarray}
Since any interval in $J_{0}$ of length $2\sigma_l^{(n)}$ is
covered by at most $3$ small intervals
from $\{I_{1}(w_{l}^{(n)}),\ldots,I_{k(w_{l}^{(n)})}(w_{l}^{(n)})\}$,
we have
\begin{eqnarray}
 \lefteqn{
  \sup_{-A_{n} \leq {\mu'} \leq A_{n}}
  \left(
   \frac{1}{n}
   R_n[{\mu'}, w_{l}^{(n)}]
   -
   3u_{0} \cdot 2w_{l}^{(n)}
  \right)
  >
  \frac{\delta}{2}
   \left(
    \log\psi(2w_{l(t)+1}^{(n)})
   \right)^{-1}
   } & & \nonumber \\
 & \Rightarrow &
  \max_{k = 1,\ldots,k(w_{l}^{(n)})}
  \left(
   \frac{1}{n}
   R_n(I_k(w_{l}^{(n)}))
   -
   u_{0} \cdot 2w_{l}^{(n)}
  \right)
  >
  \frac{1}{3} \cdot \frac{\delta}{2}
  \left(
   \log\psi(2w_{l(t)+1}^{(n)})
  \right)^{-1}.
  \label{99-1:short-interval-event:eq}
\end{eqnarray}
Note that $R_n(I_k(w_{l}^{(n)})) \sim \mathrm{Bin}(n, P_0(I_k(w_{l}^{(n)})))$
and $P_0(I_k(w_{l}^{(n)})) \leq u_{0} \cdot 2w_{l}^{(n)}$.
Therefore from (\ref{99-1:bound-k_c_n:eq}) and Okamoto's inequality (\cite{O1958}) we have
\begin{eqnarray}
 \lefteqn{
  \mathrm{Prob}
  \left(
   \max_{k = 1,\ldots,k(w_{l}^{(n)})}
   \frac{1}{n}
   \left(
    R_n(I_k(w_{l}^{(n)}))
    -
    u_{0} \cdot 2w_{l}^{(n)}
   \right)
   >
   \frac{1}{3} \cdot \frac{\delta}{2}
   \left\{
    \log{\psi(2w_{l(t)+1}^{(n)})}
   \right\}^{-1}
  \right)
  } & & \nonumber \\
  & & \hspace{1.5cm}
   \leq
   \left(
    \frac{2A_{n}}{2w_{l}^{(n)}} + M
   \right) \cdot
   \exp{
   \left[
    -2n \cdot \frac{\delta^{2}}{36}
    \left\{
     \log{\psi(2w_{l(t)+1}^{(n)})}
    \right\}^{-2}
   \right]
   }
   \nonumber \\
 & & \hspace{1.5cm}
   \leq
   \left(
    \frac{A_{n}}{\xi(M v_0/c_{n}')} + M
   \right)
   \cdot
   \exp{
   \left[
    -2n \cdot \frac{\delta^{2}}{36}
    \left\{
     \log(M v_0/c_{n}')
    \right\}^{-2}
   \right]
   }
   \label{99-1:c_n_dash:prob:eq}
   .
\end{eqnarray}
{}From (\ref{99-1:l_n:ineq:eq}),
(\ref{99-1:rarer-event:eq}),
(\ref{99-1:rarest-event:eq}),
(\ref{99-1:short-interval-event:eq}),
and (\ref{99-1:c_n_dash:prob:eq})
we obtain
\begin{eqnarray}
 & &
  \sum_{l=1}^{l_{n}}
  \mathrm{Prob}
  \left(
   \sup_{-A_{n} < {\mu'} < A_{n}}
   \frac{1}{n}
   \left(
    R_n[{\mu'}, w_{l}^{(n)}]
    -
    3u_{0} \cdot 2w_{l}^{(n)}
   \right) \cdot
   \log \psi(2w_{l+1}^{(n)})
   > \frac{\delta}{2}
  \right)
  \nonumber
\\&&  \quad
\le
\left(\frac{\xi(v_0/c_{0})}{h_{n}} + 1\right)
\cdot
\left(
    \frac{A_{n}}{\xi(M v_0/c_{n}')} + M
   \right)
   \cdot
   \exp{
   \left[
    -2n \cdot \frac{\delta^{2}}{36}
    \left\{
     \log{\left(M v_0/c_{n}'\right)}
    \right\}^{-2}
   \right]}
  \nonumber
\end{eqnarray}
When we sum this over $n$,
the resulting series on the right converges.
Hence by the Borel-Cantelli lemma and the fact
that $\delta > 0$ is arbitrary, we have
\begin{eqnarray}
 \lefteqn{
  \limsup_{n \rightarrow \infty}
  \sup_{\theta \in \Theta_{n,\mathscr{K},s,T,\tau}'}
  \left[
   \sum_{t \in \tau}
   \frac{1}{n}
   R_{n}(J_{t}(\theta)) \cdot
   \log{ H(J_{t}(\theta)) }
  \right.
  }& &
  \nonumber \\
 & & \hspace{3cm}
  \left.
   -
   3 \sum_{t \in \tau}
   {u_{0}} \cdot \xi({H(J_{t}(\theta))})
   \cdot \log{H(J_{t}(\theta))}
  \right]
  \leq 0
  \quad a.e.
  \nonumber
\end{eqnarray}
\qed

This completes the proof of theorem \ref{main-thm:thm}.

\section{Discussions}
\label{sec:discussion}

In this paper we consider the strong consistency
of MLE for mixtures of
location-scale distributions.
We treat the case that the scale parameters of
the component distributions are restricted from below
by $c_n = \exp(-n^d)$, $0 < d < 1$, and
give the regularity conditions for the strong consistency
of MLE.

As in the case of the uniform mixture in \cite{TT2003-20}, it is readily verified that
if $c_n$ decreases to zero faster than $\exp(-n)$,
then the consistency of MLE  fails.
Therefore the rate of $c_{n} = \exp(-n^d)$, $0 < d < 1$,
obtained in this paper
is almost the lower bound of the order of $c_n$ which maintains
the strong consistency.

Although we treat the univariate case in this paper, it is clear that
the result obtained in this paper
can be extended to the multivariate case
under the condition that
components
are bounded and their tails decrease to zero fast enough
if the minimum singular values of the scale matrices
of the components are restricted from below by $c_{n}$.

Finally let us consider some
sufficient conditions for the regularity conditions.
For $\theta_{m} \in \Omega_{m}$ and any positive real number $\rho$, let
\begin{eqnarray}
f_{m}(x;\theta_{m},\rho) & \equiv & \sup_{\mathrm{dist}(\theta_{m}',\theta_{m}) \leq \rho}f_m(x;\theta_{m}').
\nonumber
\end{eqnarray}
Let $\Gamma$ be any compact
subset of $\Omega_{m}$.
Consider the following two conditions.
\begin{assumption}
 For each $\theta_{m} \in \Gamma$ and sufficiently small $\rho$,
 $f_{m}(x;\theta_{m}, \rho)$ is measurable.
 \label{assumption:5}
\end{assumption}
\begin{assumption}
 For each $\theta_{m} \in \Gamma$,
 if
 $\lim_{j \rightarrow \infty}\theta_{m}^{(j)} = \theta_{m}$,
 then
 $\lim_{j \rightarrow \infty}
 f_{m}(x;\theta_{m}^{(j)}) = f_{m}(x;\theta_{m})$
 for all $x$.
 \label{assumption:6}
\end{assumption}
If assumptions~\ref{assumption:5} and \ref{assumption:6} hold,
then it is easily verified that
assumptions~\ref{assumption:2} and \ref{assumption:3}
hold.
Thus assumptions~\ref{assumption:1}, \ref{assumption:4}, \ref{assumption:5} and \ref{assumption:6} are
sufficient conditions for regularity conditions
and assumptions~\ref{assumption:5} and \ref{assumption:6} are checked more easily.
For example, finite mixture density which consists of
normal density, $t$-density and uniform density on an open interval
satisfies assumptions~\ref{assumption:1}, \ref{assumption:4}, \ref{assumption:5} and \ref{assumption:6}.

\bibliographystyle{econometrica}

redpajama_set_name: RedPajamaArXiv


Seeing is believing, as the saying goes. Without physically observing the amount of waste generated from hydraulic fracturing of unconventional oil and gas wells, it is difficult to comprehend the volume and scope of the waste produced.
The Pennsylvania Department of Environmental Protection (PADEP) makes a considerable amount of waste production data publicly available, speaking to the quantities of fluids and solids produced by 25 oil and gas operators across 25 counties. This figure, however, is only about 40% of all of the operators according to StateImpactPA. Also, complete data is not available for the 25 companies that are included, but let's dig into some waste data simply as an exercise.
In order to gain a sense for industry trends we decided to look at data pertaining to Cabot Oil and Gas Corporation, specifically, whose entire 2013 inventory of oil and gas wells were in Susquehanna County and the surrounding region. The first and second halves of 2013 contain fairly complete records for Cabot – such as well location, waste facility location, waste type, waste quantity, and disposal method. It is interesting to note that in the comments section, all but a few of the well permit sites read "Entire water fraction of waste stream recycled at a centralized treatment plant for reuse by Cabot," even for drill cuttings that were taken to a landfill.
The following analysis focuses on the waste generated by 264 Cabot wells during this period. All of Cabot's unconventional oil and gas wells in Pennsylvania during 2013 were in Susquehanna County and the surrounding region.
In the first 6 months of 2013 (Period 1), liquid waste – consisting of produced fluid, servicing fluid, hydraulic fracturing fluid (frac fluid) waste, and drilling fluid waste – totaled 745,898 barrels (Bbl) or over 30,000,000 gallons. Solid waste – or drill cuttings – totaled 51,981 tons.1 To put this into perspective, 745,898 Bbls is equivalent to the water usage requirements of about 4 wells in West Virginia.2 The 51,981 tons of drill cuttings weighs about the same as the average amount of garbage produced by 65,029 Americans per year, or 1.5 times the population of Susquehanna County. The fluid waste is also enough to fill approximately 48 Olympic swimming pools.
Period 2 (July through December) of 2013, consisting of 319 reporting wells, experienced a 77% increase in liquid waste, climbing past the 1 million Bbl mark to 1,340,143 Bbl. This figure is the equivalent of filling almost 85 Olympic swimming pools. Similarly, drill cuttings increased to 96,165 tons, almost double the amount generated in Period 1. The total amount of waste generated by Cabot for the entire year yields more than 2 million Bbl of liquid waste and nearly 150,000 tons of solid waste from drill cuttings1 – more than 130 Olympic swimming pools worth of water and a weight of solid waste equivalent to the average waste generated by more than 120,000 American per year- over 2.8 times the population pf Susquehanna County (see infographic below).
The number of unconventional oil and gas reporting wells only increased by about 20%, from 264 to 319.
Stricter reporting/enforcement by the PADEP.
Further analysis should examine the air pollution and carbon footprint generated from such extensive traffic. The miles make a difference, considering that a highly efficient tractor trailor only gets ~10 miles per gallon.
While reusing the majority of liquid waste in an effort to reduce the amount of fresh water needed for hydraulic fracturing is a positive step, transporting recycling water by truck still results in fuel used, pollutants emitted, and traffic impacts.
Cabot Oil and Gas Corp. was the second largest unconventional shale gas producer in PA behind Chesapeake Appalachia LLC, which had more than 809 reporting wells in Period 2 of 2013. With a total of 62 companies operating in PA at this time,6 the cumulative effects of waste transportation undoubtedly add up. Serious efforts should be made on the part of all oil and gas companies to reduce their waste and provide accurate and timely waste reports.


Tens of thousands of local residents and foreign visitors packed the narrow alleys of Chinatown in downtown San Francisco to join an annual mini-parade that started the official celebrations of Chinese Lunar New Year.
The parade, which began at historic St. Mary's Square in Chinatown, marched through the center of Chinatown, as kids and young students in ancient Chinese-style costumes, lion dancers, costumed stilt walkers, and drummers danced merrily in the procession to create a joyful atmosphere of festivity for people on both sides of the street.
Speaking on a stage at the destination of the mini-parade, Breed expressed greetings of New Year and warm welcome to Chinatown residents and visitors that came to share the joy and good luck brought by the upcoming Chinese lunar New Year of the Pig, which falls on Feb 5.
She called San Francisco as an "incredible city" because of its diversity, rich history and the Chinese community.
A visitor from Holland, who identified himself as Aard, said he felt amazed to be part of the celebrations. "We also have a Chinese community in Holland, but many people are only exploring Chinese restaurants," he said.
"Here in San Francisco, the people are living together and so I think that's better to have a community like this," Aard said, adding that the celebrations of the Chinese New Year are more popular in the United States than in Europe.
At a stand in the Market Street fair, Aard bought some souvenirs that bear Chinese characters with meanings of "Safe and Sound." "I want to bring them home," he said.
The mini-parade is a prelude to a major Chinatown celebration of the Lunar New Year with a more spectacular grand parade in San Francisco that has a history of more than 100 years. It will be held later this month.


text:
Why Bother Making a Will? Part 2
Posted by David Cammack on 14th February 2019
 Recently we came across another real example of why you should make a will sooner rather than later. Naturally we've anonymised this story.
The situation was that Jack and Jill had married. Jill had children from a previous marriage. The children were now all adults, so no longer counted as being her dependents. Jill had not made a will. Sadly, she had developed dementia and no longer had the capacity to make a will for herself. However, she had made a Lasting Power of Attorney before she lost mental capacity.
We were asked if, acting under the Lasting Power of Attorney, her attorney could make a will for her while she was alive. The answer is "no".
We were asked what could be done, if anything, to ensure Jill's children received a reasonable inheritance.
What if mum dies first?
The main asset, the house, was in joint names on a "joint tenancy" basis, as is commonly the case. If Jill died first, most of the estate would pass to her second husband, with Jill's half of the house passing by "survivorship" automatically to Jack. Cash in joint bank accounts would also pass to Jack automatically. Jill's personal effects would also go to Jack. Under the intestacy rules, after the share of the house, all the personal effects and the joint bank accounts, a further £250,000 of value would be inherited by Jack before the children could inherit anything. Even then, the rest would be split half-and-half between the step-father and the children.
In this case the estate was not large enough to guarantee that anything would be left to Jill's children if Jill died first.
What happens if their step-father dies first?
But what if Jack dies first? He still had mental capacity and he had made a will. The contents of the will were unknown, but Jill's children were assuming it left nothing to them directly. As they would not be dependents, they could not dispute Jack's estate if he chose to leave them nothing in his will.
Their only hope was that Jack would die first and leave a part of his estate to Jill in his will, and therefore if Jill died second, they could then inherit all of Jill's estate when she died under the intestacy rules (there then being no surviving spouse to take the majority of her estate by that stage). Assuming Jack did not change the basis of the joint tenancy of the property, at least his half of it would pass by survivorship to Jill when he dies. It would not even be affected by his will.
Not a nice situation. This could all have been avoided if Jill had made a will at any stage after she married Jack and before she lost mental capacity later in life. She had many years in which she could have done so, but sadly kept putting it off.
Severing the joint tenancy
Technically Jill's attorney could sever the joint tenancy while she is alive. This would mean that if Jill died first, her share would not automatically pass to Jack. Also at least it would count towards his entitlement to the first £250,000 of her estate. If the estate were large enough, Jill's children might then get something. There are however problems with the attorney's doing this.
Firstly, the duty of the attorney is to act in the best interests of Jill. This action arguably gave Jill no benefit. Jack could therefore dispute it and the Office of the Public Guardian might reverse it.
Secondly, if Jack died first (and had not severed the tenancy himself before dying), then Jill's attorney, having severed the tenancy, might make Jill miss out on inheriting Jack's half of the house by survivorship, and potentially allow him to pass it to someone else via his will. Jill could dispute his making no provision for her in his will (as a dependent). However, this would inevitably lead to legal costs and distress for all concerned. All-in-all a non-starter.
How can you avoid such hardship for your family?
With Legalo's range of wills templates for single people costing £24.95 and for couples at £29.95, there's never been a better time to get a will in place. You don't have to have a solicitor to make a valid will – you can do it all yourself. The guide that comes with the will template will make it very easy for you to do. It will only take you minutes to have peace of mind.

redpajama_set_name: RedPajamaCommonCrawl


Senior officials from Nepal, India, Bhutan and Bangladesh are scheduled to be in New Delhi for a two-day meeting beginning on Friday to push for sub-regional cooperation in energy and connectivity.
Nepal did not participate in the first meeting of the series that took place in Dhaka, Bangladesh, in 2013.
The meeting will focus on exploring energy and water management projects in the region, while transport, transit, trade and connectivity issues are also likely to be taken up.
The Nepali delegation will be led by Joint-secretary at the Foreign Ministry Prakash Subedi, the division head for South Asia and India.
According to a senior ministry official, the meeting will explore and identify projects that can be cooperated among the four nations, more so after the Saarc Framework Agreement of Energy was signed during the 18th Saarc Summit held in Kathmandu last November.
"Nepal, India and Bangladesh can work together in the energy sector while there are prospects of connecting the four nations to boost trade," the official said.
Following the framework agreement signed between the two countries during the visit of Indian Prime Minister Manmohan Singh to Bangladesh in September 2011, the first meeting between India, Bangladesh and Bhutan under the working groups of Water and Power, and Transit and Connectivity was held in April 2013.
The meeting discussed enhancing cooperation in sharing the water of common rivers and utilising their energy potential. The second working group on transit and connectivity emphasised on implementing measures for the upgradation of road and railway networks.
In New Delhi, officials will look into immediate trilateral cooperation in trade and transit field between Nepal, India and Bangladesh through the Fulbari-Banglabandh stretch. Officials said that once the regional Saarc grid comes into effect, the entire region should be connected. Nepal is also hosting a ministerial-level symposium on energy cooperation of these four countries in April to further stimulate the sub-regional push.
This is a good beginning, said Tara Pokhrel, the ministry spokesperson. "There is optimism and renewed hope that Saarc can start implementing its commitments. Energy is the sector we can work on immediately," he said.


This flavourful Winter Vegetable Stew is made in one pot. this combination is perfect for a hearty and great dinner time.
Peel and wash the vegetables, cut into cubes or strips and blanch the cabbage.
In a cast iron pan fry in the duck fat and in order, shallots, garlic then all the vegetables except the cabbage, ham and at the end add cabbage. Season, then add the herbs and broth.
Bring to a boil, cover and bake at 150 ° C (320°F) for 35 minutes or cook over low heat, stirring from time to time. Serve immediately.