text:
\section{Introduction}
\label{sec:Intro}

A pairwise Boltzmann machine (PBM)~\cite{Ackley_etal1985} and its variants,
such as a higher-order Boltzmann machine~\cite{HBM1986}, restricted Boltzmann machine (RBM)~\cite{CD2002,RBM1986}, and deep Boltzmann machine (DBM)~\cite{DBM2009},
are one of the most fundamental and important models in the field of probabilistic machine learning.
Except for some special cases, the inference and learning of PBMs are computationally difficult because they include a multiple summation (or integration) over its all variables.
Therefore, the development of approximations for them has attracted attention in the field.
For PBM learning, various methods were proposed, such as,
mean-field learning methods (e.g., the mean-field approximation~\cite{Kappen1998},
the Bethe approximation (or loopy belief propagation)~\cite{Yasuda2006,Yasuda&Tanaka2009,Federico2012,LBP2012,Cyril2013},
the Plefka expansion~\cite{TTanaka1998,Monasson2009}),
maximum pseudo-likelihood estimation (MPLE)~\cite{PL1975,Hyvarinen2006},
contrastive divergence~\cite{CD2002},
ratio matching (RM)~\cite{RM2007}, and minimum probability flow (MPF)~\cite{MPF2011}.

Evaluating the expectations of the variables in PBMs is critical for PBM learning.
This evaluation is generally NP-hard owing to the multiple summation.
A Monte Carlo integration (MCI) method is the simplest way to approximate the expectations,
in which they are approximated by the sample average over the sample points obtained by using a sampling method (e.g., Gibbs sampling) on the PBM.
An effective MCI, called \textit{spatial Monte Carlo integration} (SMCI), was proposed~\cite{Yasuda2015}.

Here, the basic concept of SMCI is informally explained.
Imagine a PBM, $P(\bm{x})$, defined on a undirected (connected) graph $G(\mcal{V}, \mathbb{F}_2)$ with $n$ vertices,
where $\bm{x}$ is the set of $n$ variables, $\mcal{V}$ is the set of indices of the vertices, and $\mathbb{F}_2$ is the set of undirected edges.
For simplicity, the variables are all $\{-1,+1\}$-binary.
Consider the expectation of $x_i$, i.e., $\ave{x_i} = \sum_{\bm{x}}x_i P(\bm{x})$.
The exact evaluation of this expectation costs $O(2^n)$.
In SMCI, this expectation is approximated as follows.
Suppose that the sample set, $\mathbb{S}$, consists of $M$ sample points generated by a sampling method.
Take a connected region (or subgraph), $\mcal{A}$, covering vertex $i$, namely, $\{i\} \subseteq   \mcal{A} \subseteq \mcal{V}$.
For the region $\mcal{A}$, the target expectation is approximated as
\begin{align}
\ave{x_i} \approx \frac{1}{M}\sum_{\ell= 1}^M \sum_{\bm{x}_{\mcal{A}}}x_i P \big(\bm{x}_{\mcal{A}} \mid \bm{x}_{\partial \mcal{A}} = (\text{$\ell$th sample point}) \big),
\label{eqn:concept_SMCI}
\end{align}
where $\bm{x}_{\mcal{A}}$ is the set of variables in $\mcal{A}$, $\bm{x}_{\partial \mcal{A}}$ is the set of variables in the neighborhood of $\mcal{A}$,
and $P(\bm{x}_{\mcal{A}} \mid \bm{x}_{\partial \mcal{A}} )$ is the conditional distribution of the PBM.
In SMCI, the target index, $i$, is referred to as the \textit{target region};
$\mcal{A}$ and $\partial \mcal{A}$ are referred to as the \textit{sum region} and the \textit{sample region}, respectively.
The computational cost of equation (\ref{eqn:concept_SMCI}) is $O(M2^{|\mcal{A}|})$, which can be evaluated as long as the size of $\mcal{A}$ is not large.
Figure \ref{fig:region_sheme_SMCI} illustrates the scheme of the regions of SMCI.
The formal formulation of equation (\ref{eqn:concept_SMCI}) will be presented in section \ref{sec:GSMCI}.

\begin{figure}[tb]
\begin{center}
\includegraphics[height=3.5cm]{region_sheme.eps}
\end{center}
\caption{Illustration of the target, sum, and sample regions of SMCI.}
\label{fig:region_sheme_SMCI}
\end{figure}

The primary concern of SMCI is how to determine the sum region.
The original SMCI~\cite{Yasuda2015} determined the sum region
so as to cover up to $(k-1)$th nearest neighbors of a specific target region $\mcal{T}$.
This was referred to as the $k$th-order SMCI ($k$-SMCI) method.
For the approximation accuracy of the $k$-SMCI method, two important statements were proved~\cite{Yasuda2015}:
the $k$th-order SMCI is statistically more accurate than both (i) the standard MCI method for any $k \geq 1$
and (ii) the $(k-1)$-SMCI method.
These statements guarantee that a more accurate approximation can be obtained by increasing the value of $k$ (i.e., the level of approximation of SMCI).
The 1-SMCI method (the simplest $k$-SMCI method) can be applied to any graph as long as the size of the target region is not large,
because the sum region is identified to the target region in this simplest case.
However, the 2-SMCI method is not always usable; the sum region can include $O(n)$ variables in a dense graph.
This is a major drawback of the $k$-SMCI method.

In the original SMCI~\cite{Yasuda2015}, as the level of approximation increases,
the sum region is systematically expanded according to the neighboring relationship among vertices;
and this causes the problem mentioned above.
A more flexible setting of the sum region is desired; for example,
the sum region covers a part of the first-nearest neighbors of the target region,
which is called the semi-second-order SMCI (s2-SMCI) method in this paper (cf. section \ref{sec:s2-SMCI}).
However, accuracy bounds for such sum regions were not clarified.
Intuitively, a larger sum region can be more accurate.
More concretely, for a specific target region $\mcal{T}$,
suppose that there are two different regions such that $\mcal{T}\subseteq \mcal{U}_1 \subseteq \mcal{U}_2$;
in this case, SMCI using $\mcal{U}_2$ as the sum region is more accurate.
This intuition is in fact true (cf. Theorem \ref{theo:GSMCI}).
This is the first contribution of this study.
This type of SMCI is referred to as the generalized SMCI (GSMCI) method in this paper.
From this fact, one can adaptively choose the sum region according to the structure of graph with a clear bound on the approximation accuracy.
In section \ref{sec:SMCI}, the $k$-SMCI method proposed in the previous study is briefly explained.
The GSMCI method is introduced in section \ref{sec:GSMCI}.
In these sections, the $k$-SMCI and GSMCI methods are formulated on a higher-order Markov random field (HMRF),
which is a generalized Markov random field and includes the PBM as a special case.
The application of the SMCI methods (the $k$-SMCI and GSMCI methods) to the PBM and their numerical validation are presented in section \ref{sec:application_PBM}.

The second contribution of this study is on PBM learning.
The $k$-SMCI method was applied to PBM learning~\cite{Yasuda2015,SMCI2018} and to another learning problem~\cite{Yasuda2019}.
In PBM learning, the method based on the 1-SMCI method was superior to the other known learning methods (MPLE, RM, and MPF)~\cite{SMCI2018}.
In the original learning method~\cite{Yasuda2015,SMCI2018}, the variables in the sample region were fixed by the given training set,
leading to an useful deterministic algorithm.
However, the accuracy of the learning cannot be improved without increasing the level of approximation of SMCI (i.e., increasing the value of $k$) in this method.
In this paper, a new learning method is proposed by combining SMCI with persistent contrastive divergence (PCD)~\cite{PCD2008}.
This proposed method allows the accuracy of the learning to be improved without increasing the level of approximation of SMCI.
The proposed learning method (with its pseudocode) and its numerical validations are described in section \ref{sec:PBM_learning}.

\section{Spatial Monte Carlo Integration}
\label{sec:SMCI}

In this section, the original SMCI, i.e., the $k$-SMCI method, is explained.

\subsection{Higher-order Markov random field}
\label{sec:HOMRF}

Consider a higher-order MRF (HMRF) consisting of $n$ random variables, $\bm{x} := \{ x_i \in \mcal{X}_i \mid i \in \mcal{V}\}$,
where $\mcal{X}_i$ is the sample space of $x_i$ and $\mcal{V}:= \{1,2,\ldots, n\}$ is the set of indices of the variables,
which is defined by
\begin{align}
P(\bm{x}):=\frac{1}{Z} \exp\Big( \sum_{\mcal{C} \in \mathbb{F}} \phi_{\mcal{C}} (\bm{x}_{\mcal{C}})\Big),
\label{eqn:HMRF}
\end{align}
where $\mcal{C}$ denotes a clique, i.e., $\mcal{C}\subseteq \mcal{V}$, and $\mathbb{F}$ denotes a family of cliques.
A HMRF is regarded as a probabilistic graphical model on an undirected hypergraph,
in which $\mcal{V}$ is regarded as the set of vertices, and $\mathbb{F}$ is regarded as the set of hyperedges in the hypergraph.
The function $\phi_{\mcal{C}}(\bm{x}_{\mcal{C}})$ denotes a potential function on ${\mcal{C}}$, where $\bm{x}_{\mcal{C}} := \{x_i \mid i \in \mcal{C} \subseteq \mcal{V}\}$.
In equation (\ref{eqn:HMRF}), $Z$ denotes the partition function defined by
\begin{align*}
Z: = \sum_{\bm{x}} \exp\Big( \sum_{\mcal{C} \in \mathbb{F}} \phi_{\mcal{C}} (\bm{x}_{\mcal{C}})\Big),
\end{align*}
where $\sum_{\bm{x}} := \sum_{x_1 \in \mcal{X}_1}\sum_{x_2 \in \mcal{X}_2}\cdots \sum_{x_n \in \mcal{X}_n}= \prod_{i \in \mcal{V}}\sum_{x_i \in \mcal{X}_i}$
denotes the summation over all the possible realizations of $\bm{x}$.
It should be noted that when $\mcal{X}_i$ is a continuous space, the corresponding sum $\sum_{x_i \in \mcal{X}_i}$ is replaced with the integration $\int_{\mcal{X}_i} dx_i$.
When $\phi_{\mcal{C}}(\bm{x}_{\mcal{C}}) = w_{\mcal{C}} \prod_{i \in \mcal{C}}x_i$, the HMRF is identical to a generalization of a higher-order Boltzmann machine~\cite{HBM1986}.
In particular,
when $\mathbb{F} = \mathbb{F}_1 \cup \mathbb{F}_2$, where $\mathbb{F}_1:= \{\{i\} \mid i \in \mcal{V}\}$
and $\mathbb{F}_2$ is a family of pairs of indices (i.e., $\mathbb{F}_2:= \{\{i,j\}\}$), and $\phi_{\mcal{C}}(\bm{x}_{\mcal{C}}) = w_{\mcal{C}} \prod_{i \in \mcal{C}}x_i$,
equation (\ref{eqn:HMRF}) is identical to a familiar PBM~\cite{Ackley_etal1985}:
\begin{align}
P(\bm{x}):=\frac{1}{Z} \exp\Big(\sum_{i \in \mcal{V}}w_i x_i +  \sum_{\{i,j\} \in \mathbb{F}_2} w_{i,j} x_ix_j\Big),
\label{eqn:pairwiseMRF}
\end{align}
where $w_{i,j}$ is identical to $w_{j,i}$. $w_i$ and $w_{i,j}$ are called the \textit{bias} and \textit{interaction} parameters, respectively.

Here, consider the expectation of a function over specific target variables, $\bm{x}_{\mcal{T}}$, where $\mcal{T}\subseteq \mcal{V}$, on the HMRF, which is
\begin{align}
\ave{f(\bm{x}_{\mcal{T}})} := \sum_{\bm{x}}f(\bm{x}_{\mcal{T}}) P(\bm{x}) = \sum_{\bm{x}_{\mcal{T}}}f(\bm{x}_{\mcal{T}}) P(\bm{x}_\mcal{T}),
\label{eqn:target_expectation}
\end{align}
where $P(\bm{x}_{\mcal{T}}) = \sum_{\bm{x} \setminus \bm{x}_{\mcal{T}}}P(\bm{x})$ is the marginal distribution of the HMRF.
However, in general, the evaluation of this expectation is computationally difficult owing to the multiple summation.
SMCI, which is an extension of the standard MCI method, was proposed to efficiently approximate the expectation~\cite{Yasuda2015}.

Hereafter, we assume that the size of $\mcal{T}$ is not large and a sum over $\bm{x}_{\mcal{T}}$ can be numerically evaluated.

\subsection{Adjacency relations in HMRF}

For a detailed explanation of the $k$-SMCI method, an adjacency relation among the indices in $\mcal{V}$ must be defined.
If a clique $\mcal{C}$ involves both $i$ and $j$, both indices are regarded as the first-nearest-neighboring indices of each other
(or, in other words, both indices are connected).
If no cliques involve both $i$ and $j$ and two different cliques involving $i$ and $j$, respectively, overlap,
both indices are regarded as the second-nearest-neighboring indices of each other (see figure \ref{fig:adjacency_relations_HMRF}).
\begin{figure}[tb]
\begin{center}
\includegraphics[height=2.5cm]{adjacency_relations_HMRF.eps}
\end{center}
\caption{Adjacency relations among the indices. For example, $i$ and $j$ are the first-nearest-neighboring indices of each other;
$i$ and $l$ are the second-nearest-neighboring indices of each other.}
\label{fig:adjacency_relations_HMRF}
\end{figure}

Similar to the above, neighboring regions of a target region $\mcal{T} \subseteq \mcal{V}$ are defined as follows.
For the target region $\mcal{T}$, the first-nearest-neighboring region of $\mcal{T}$, $\mcal{N}_1(\mcal{T}) \subseteq \mcal{V}$, is defined as
\begin{align*}
\mcal{N}_1(\mcal{T}) := \{i \mid i \in \mcal{C} \in \mathbb{F}(\mcal{T}), \> i \not\in \mcal{T}\},
\end{align*}
where $\mathbb{F}(\mcal{A})$ is a subset of $\mathbb{F}$ that is the family of the cliques which overlaps the assigned set $\mcal{A} \subseteq \mcal{V}$ i.e.,
$\mathbb{F}(\mcal{A}) := \{\mcal{C} \mid \mcal{C} \in \mathbb{F},\> \mcal{C} \cap \mcal{A} \neq \emptyset\}$.
In other words, the indices in $\mcal{N}_1(\mcal{T})$ do not belong to $\mcal{T}$ and simultaneously are the first-nearest neighbors of the indices in $\mcal{T}$.
The second-nearest-neighboring region of $\mcal{T}$, $\mcal{N}_2(\mcal{T}) \in \mcal{V}$, is defined as
\begin{align*}
\mcal{N}_2(\mcal{T}) := \{i \mid i \in \mcal{C} \in \mathbb{F}(\mcal{N}_1(\mcal{T})), \> i \not\in \mcal{T} \cup \mcal{N}_1(\mcal{T})\}.
\end{align*}
This means that the indices in $\mcal{N}_2(\mcal{T})$ do not belong to $\mcal{T}$ or $\mcal{N}_1(\mcal{T})$
and simultaneously are the second-nearest neighbors of the indices in $\mcal{T}$.
In a similar manner, the $k$th-nearest-neighboring region of $\mcal{T}$, $\mcal{N}_k(\mcal{T}) \subseteq \mcal{V}$, is defined as
\begin{align}
\mcal{N}_k(\mcal{T}) := \{i \mid i \in \mcal{C} \in \mathbb{F}(\mcal{N}_{k-1}(\mcal{T})), \> i \not\in \mcal{R}_{k-1}(\mcal{T})\},
\label{eqn:R_k(T)}
\end{align}
where $\mcal{R}_{k}(\mcal{T}):= \bigcup_{r=0}^k \mcal{N}_r(\mcal{T})$ and $\mcal{N}_0(\mcal{T}):= \mcal{T}$ i.e.,
$\mcal{R}_{k}(\mcal{T})$ is the region that covers the regions $\mcal{T}, \mcal{N}_1(\mcal{T}), \ldots,\mcal{N}_k(\mcal{T})$.

The adjacency relations can be easily understand in a pairwise case in which $\mathbb{F} = \mathbb{F}_1 \cup \mathbb{F}_2$.
In this case, the adjacency relation can be viewed as an undirected graph $G(\mcal{V}, \mathbb{F}_2)$,
and the adjacency relations represented above are identical to the standard adjacency relations in the graph.
In figure \ref{fig:adjacency_relations_MRF}, the examples of the adjacency relations in a pairwise case, in which $G(\mcal{V}, \mathbb{F}_2)$ is a square grid graph, are illustrated.
\begin{figure}[tb]
\begin{center}
\includegraphics[height=4cm]{adjacency_relations_MRF.eps}
\end{center}
\caption{Examples of the adjacency relations in a pairwise case:
(a) when $\mcal{T} = \{13\}$, $\mcal{N}_1(\mcal{T}) =\{ 8,12,14,18\}$ and $\mcal{N}_2(\mcal{T}) =\{ 3,7,9,11,15,17,19,23\}$,
and (b) when $\mcal{T} =\{12,13\}$, $\mcal{N}_1(\mcal{T}) =\{7,8,11,14,17,18\}$.}
\label{fig:adjacency_relations_MRF}
\end{figure}

\subsection{$k$th-order SMCI method for HMRF}
\label{sec:k-SMCI}

The spatial Markov property of the HMRF ensures that the conditional distribution
$P(\bm{x}_{\mcal{R}_{k-1}(\mcal{T})} \mid \bm{x} \setminus \bm{x}_{\mcal{R}_{k-1}(\mcal{T})})$ can be expressed as
\begin{align*}
P(\bm{x}_{\mcal{R}_{k-1}(\mcal{T})} \mid \bm{x} \setminus \bm{x}_{\mcal{R}_{k-1}(\mcal{T})})
= P(\bm{x}_{\mcal{R}_{k-1}(\mcal{T})} \mid \bm{x}_{\mcal{N}_{k}(\mcal{T})}),
\end{align*}
where
\begin{align*}
P(\bm{x}_{\mcal{R}_{k-1}(\mcal{T})} \mid \bm{x}_{\mcal{N}_{k}(\mcal{T})}) \propto \exp\Big( \sum_{\mcal{C} \in \mathbb{F}(\mcal{R}_{k-1}(\mcal{T}))} \phi_{\mcal{C}} (\bm{x}_{\mcal{C}})\Big)
\end{align*}
is also an HMRF.
Therefore, the expectation in equation (\ref{eqn:target_expectation}) can be expressed as
\begin{align}
\ave{f(\bm{x}_{\mcal{T}})} = \sum_{\bm{x}_{\mcal{R}_k(\mcal{T})}}f(\bm{x}_{\mcal{T}}) P(\bm{x}_{\mcal{R}_{k-1}(\mcal{T})} \mid \bm{x}_{\mcal{N}_{k}(\mcal{T})}) P(\bm{x}_{\mcal{N}_{k}(\mcal{T})}),
\label{eqn:SMCI_origin}
\end{align}
where $P(\bm{x}_{\mcal{N}_{k}(\mcal{T})})$ is the marginal distribution of $P(\bm{x})$.

Suppose that $M$ i.i.d. sample points are drawn from $P(\bm{x})$: $\mathbb{S}:=\{ \mbf{s}^{(\ell)} \mid \ell = 1,2,\ldots, M\}$,
where $\mbf{s}^{(\ell)}:= \{\mrm{s}_i^{(\ell)} \in \mcal{X}_i \mid i \in \mcal{V}\}$ is the $\ell$th sample point.
In the $k$-SMCI method~\cite{Yasuda2015}, the expectation in equation (\ref{eqn:SMCI_origin}) is approximated by
\begin{align}
m_{\mcal{T}}^{(k)}(\mathbb{S})
:= \frac{1}{M}\sum_{\ell=1}^M \sum_{\bm{x}_{\mcal{R}_{k-1}(\mcal{T})}}f(\bm{x}_{\mcal{T}}) P\big(\bm{x}_{\mcal{R}_{k-1}(\mcal{T})} \mid \mbf{s}_{\mcal{N}_{k}(\mcal{T})}^{(\ell)} \big),
\label{eqn:k-SMCI}
\end{align}
where $\mbf{s}_{\mcal{A}}^{(\ell)} := \{\mrm{s}_{i}^{(\ell)} \mid i \in \mcal{A}\}$.
In this approximation, $\mcal{R}_{k-1}(\mcal{T})$ is regarded as the sum region, and $\mcal{N}_{k}(\mcal{T})$ is regarded as the sample region.
Equation (\ref{eqn:k-SMCI}) is obtained by replacing the marginal distribution $P(\bm{x}_{\mcal{N}_{k}(\mcal{T})})$ in equation (\ref{eqn:SMCI_origin})
with the corresponding empirical distribution of the given sample points, which is defined by
\begin{align*}
Q_{\mathbb{S}}(\bm{x}_{\mcal{N}_{k}(\mcal{T})}):= \frac{1}{M}\sum_{\ell = 1}^M \delta\big(\bm{x}_{\mcal{N}_{k}(\mcal{T})}, \mbf{s}_{\mcal{N}_{k}(\mcal{T})}^{(\ell)}\big),
\end{align*}
where $\delta$ is the Kronecker (or Dirac) delta function:
\begin{align*}
\ave{f(\bm{x}_{\mcal{T}})} \approx \sum_{\bm{x}_{\mcal{R}_k(\mcal{T})}}f(\bm{x}_{\mcal{T}}) P(\bm{x}_{\mcal{R}_{k-1}(\mcal{T})} \mid \bm{x}_{\mcal{N}_{k}(\mcal{T})}) Q_{\mathbb{S}}(\bm{x}_{\mcal{N}_{k}(\mcal{T})}) = m_{\mcal{T}}^{(k)}(\mathbb{S}).
\end{align*}
The $k$-SMCI method is usable when the sum over $\bm{x}_{\mcal{R}_{k-1}(\mcal{T})}$ is computable.

\subsection{Asymptotic analysis of $k$-SMCI method}
\label{sec:k-SMCI_asymptotic}

Because the sample points $\mbf{s}^{(\ell)}$ are i.i.d. random variables,
\begin{align*}
\rho_{\mcal{T}}^{(k)}(\mbf{s}_{\mcal{N}_{k}(\mcal{T})}^{(\ell)}):=\sum_{\bm{x}_{\mcal{R}_{k-1}(\mcal{T})}}f(\bm{x}_{\mcal{T}}) P\big(\bm{x}_{\mcal{R}_{k-1}(\mcal{T})} \mid \mbf{s}_{\mcal{N}_{k}(\mcal{T})}^{(\ell)} \big)
\end{align*}
are also regarded as i.i.d. random variables.
Therefore, from the result of the central limit theorem,
the distribution of $m_{\mcal{T}}^{(k)}(\mathbb{S}) = M^{-1} \sum_{\ell = 1}^M \rho_{\mcal{T}}^{(k)}(\mbf{s}_{\mcal{N}_{k}(\mcal{T})}^{(\ell)})$ is asymptotically close to the Gaussian
with mean
\begin{align*}
\mu_{\mcal{T}}^{(k)}:= \Big(\prod_{\ell = 1}^M \sum_{\mbf{s}^{(\ell)}}P(\mbf{s}^{(\ell)})\Big) m_{\mcal{T}}^{(k)}(\mathbb{S})
\end{align*}
and variance
\begin{align*}
v_{\mcal{T}}^{(k)} := \Big(\prod_{\ell = 1}^M \sum_{\mbf{s}^{(\ell)}}P(\mbf{s}^{(\ell)})\Big) m_{\mcal{T}}^{(k)}(\mathbb{S})^2  - \big(\mu_{\mcal{T}}^{(k)}\big)^2
=O(M^{-1}),
\end{align*}
for a sufficiently large $M$, where $\sum_{\mbf{s}^{(\ell)}} := \prod_{i \in \mcal{V}}\sum_{\mrm{s}_i^{(\ell)} \in \mcal{X}_i}$
and $P(\mbf{s}^{(\ell)})$ is the HMRF in equation (\ref{eqn:HMRF}).
The asymptotic mean is equivalent to the exact expectation i.e., $\mu_{\mcal{T}}^{(k)}=\ave{f(\bm{x}_{\mcal{T}})}$.
Therefore, $m_{\mcal{T}}^{(k)}(\mathbb{S})$ converges to $\ave{f(\bm{x}_{\mcal{T}})}$ as $M$ approaches infinity, and its variance vanishes at a speed of $O(M^{-1})$.

From the perspective of statistics, a method having a smaller asymptotic variance is preferable.
For the asymptotic variance $v_{\mcal{T}}^{(k)}$, the following theorem was obtained~\cite{Yasuda2015}.
\begin{theorem}
In the HMRF expressed in equation (\ref{eqn:HMRF}), the inequality relation
$v_{\mcal{T}}\geq v_{\mcal{T}}^{(1)}\geq v_{\mcal{T}}^{(2)}\geq v_{\mcal{T}}^{(3)}\geq \cdots \geq 0$ always holds
for any $M$ and for any choice of target region $\mcal{T}$, where $v_{\mcal{T}}$ is the asymptotic variance of the standard MCI method.
\label{theo:k-SMCI}
\end{theorem}
This theorem states that, for a sufficient large $M$, the $k$-SMCI method is statistically more accurate than the standard MCI method for any $k\geq 1$
and that a higher-order SMCI method is statistically more accurate than any lower-order method.

\section{Generalization of SMCI}
\label{sec:GSMCI}

In this section, the GSMCI method is introduced.
Here, for a region $\mcal{A}$, satisfying $\mcal{T} \subseteq \mcal{A} \subseteq \mcal{V}$, and for a sample set $\mathbb{S}$, consider an SMCI defined as
\begin{align}
m_{\mcal{T}}(\mcal{A}; \mathbb{S}):=
 \frac{1}{M}\sum_{\ell=1}^M\sum_{\bm{x}_{\mcal{A}}}f(\bm{x}_{\mcal{T}}) P\big(\bm{x}_{\mcal{A}} \mid \mbf{s}_{\partial \mcal{A}}^{(\ell)} \big),
\label{eqn:GSMCI}
\end{align}
where $\partial \mcal{A}$ denotes the first-nearest-neighboring region of region $\mcal{A}$:
$\partial \mcal{A}:=\{i \mid i \in \mcal{C} \in \mathbb{F}(\mcal{A}),\> i \not\in \mcal{A}\}$.
The conditional distribution in equation (\ref{eqn:GSMCI}) is expressed as
\begin{align}
P\big(\bm{x}_{\mcal{A}} \mid \bm{x}_{\partial \mcal{A}} \big)
\propto \exp\Big( \sum_{\mcal{C} \in \mathbb{F}(\mcal{A})} \phi_{\mcal{C}} (\bm{x}_{\mcal{C}})\Big).
\label{eqn:conditional-dist_GSMCI}
\end{align}
When $\mcal{A} = \mcal{R}_{k-1}(\mcal{T})$, $\partial \mcal{A}$ is identical to $\mcal{N}_k(\mcal{T})$;
therefore, equation (\ref{eqn:GSMCI}) is equivalent to equation (\ref{eqn:k-SMCI}) in this case.
Therefore, equation (\ref{eqn:GSMCI}) is regarded as a generalization of the $k$-SMCI method.
By analogy with the $k$-SMCI method, it is expected that the approximation accuracy of equation (\ref{eqn:GSMCI}) will increase as the sum region $\mcal{A}$ becomes larger.
In fact, the following argument justifies this expectation.

Consider two sum regions $\mcal{U}_1$ and $\mcal{U}_2$ that satisfy $\mcal{T} \subseteq \mcal{U}_1 \subseteq \mcal{U}_2$.
For the two sum regions, based on equation (\ref{eqn:GSMCI}),
the two approximations $m_{\mcal{T}}(\mcal{U}_1; \mathbb{S})$ and $m_{\mcal{T}}(\mcal{U}_2; \mathbb{S})$
are considered for the purpose of approximating $\ave{f(\bm{x}_{\mcal{T}})}$.
With a similar argument to that obtained in section \ref{sec:k-SMCI_asymptotic},
the asymptotic properties of both approximations are analyzed as follows.
For a sufficiently large $M$, the distributions of both $m_{\mcal{T}}(\mcal{U}_1; \mathbb{S})$ and $m_{\mcal{T}}(\mcal{U}_2; \mathbb{S})$
are asymptotically close to the different Gaussians
with mean $\mu(\mcal{U}_1)$ and variance $v(\mcal{U}_1)$ and with mean $\mu(\mcal{U}_2)$ and variance $v(\mcal{U}_2)$, respectively.
These asymptotic means and variances are defined as
\begin{align}
\mu(\mcal{A})&:= \Big(\prod_{\ell = 1}^M \sum_{\mbf{s}^{(\ell)}}P(\mbf{s}^{(\ell)})\Big) m_{\mcal{T}}(\mcal{A}; \mathbb{S}) = \ave{f(\bm{x}_{\mcal{T}})},
\label{eqn:mu-GSMCI}\\
v(\mcal{A}) &:= \Big(\prod_{\ell = 1}^M \sum_{\mbf{s}^{(\ell)}}P(\mbf{s}^{(\ell)})\Big) m_{\mcal{T}}(\mcal{A}; \mathbb{S})^2  - \mu(\mcal{A})^2,
\label{eqn:v-GSMCI}
\end{align}
for a sum region $\mcal{T} \subseteq \mcal{A} \subseteq \mcal{V}$.
Equation (\ref{eqn:v-GSMCI}) is rewritten as
\begin{align}
v(\mcal{A})= \frac{1}{M}\Big( \sum_{\bm{x}_{\partial \mcal{A}}} \rho_{\mcal{T}}(\mcal{A}; \bm{x}_{\partial \mcal{A}})^2P(\bm{x}_{\partial \mcal{A}})
- \ave{f(\bm{x}_{\mcal{T}})}^2\Big),
\label{eqn:v-GSMCI_trans}
\end{align}
where
\begin{align}
\rho_{\mcal{T}}(\mcal{A}; \bm{x}_{\partial \mcal{A}}):=
\sum_{\bm{x}_{\mcal{A}}}f(\bm{x}_{\mcal{T}}) P(\bm{x}_{\mcal{A}} \mid \bm{x}_{\partial\mcal{A}} ).
\label{eqn:rho-GSMCI}
\end{align}
Here, $P(\bm{x}_{\partial \mcal{A}})$ and $P(\bm{x}_{\mcal{A}} \mid \bm{x}_{\partial\mcal{A}} )$ are the marginal and conditional distributions of the HMRF of equation (\ref{eqn:HMRF}),
respectively.
Therefore, it is found that the two approximations $m_{\mcal{T}}(\mcal{U}_1; \mathbb{S})$ and $m_{\mcal{T}}(\mcal{U}_2; \mathbb{S})$
converge to the true expectation $\ave{f(\bm{x}_{\mcal{T}})}$ as $M$ approaches infinity, and their variances vanish at speeds of $O(M^{-1})$, as in the $k$-SMCI method.

For the asymptotic variances $v(\mcal{U}_1)$ and $v(\mcal{U}_2)$, the following theorem is obtained.
\begin{theorem}
In the HMRF expressed in equation (\ref{eqn:HMRF}), for $\mcal{T} \subseteq \mcal{U}_1 \subseteq \mcal{U}_2$, the inequality relation
$v(\mcal{U}_1)\geq v(\mcal{U}_2)\geq 0$ always holds
for any $M$ and for any choice of target region $\mcal{T}$.
\label{theo:GSMCI}
\end{theorem}
The proof of this theorem is described in \ref{app:proof}.
This states that $m_{\mcal{T}}(\mcal{U}_2; \mathbb{S})$ is statistically more accurate than $m_{\mcal{T}}(\mcal{U}_1; \mathbb{S})$ for a sufficient large $M$.
It is noteworthy that this theorem includes the statement of theorem \ref{theo:k-SMCI} as its corollary.

Equation (\ref{eqn:GSMCI}) is referred to as the GSMCI method in this paper.
Because equation (\ref{eqn:GSMCI}) is identical to the 1-SMCI method when $\mcal{T} = \mcal{A}$,
the GSMCI method is statistically more accurate than the standard MCI method for any choice of $\mcal{A}$ satisfying $\mcal{T} \subseteq \mcal{A} \subseteq \mcal{V}$.
From theorem \ref{theo:GSMCI}, any sum region can be freely selected in equation (\ref{eqn:GSMCI}), and
it is guaranteed that the approximation accuracy of the GSMCI method will monotonically increases as the size of the selected sum region increases.
The GSMCI method is usable when the sum over $\bm{x}_{\mcal{A}}$ is computable.

\section{Application to PBMs}
\label{sec:application_PBM}

In this section, the SMCI methods (the $k$-SMCI and GSMCI methods) for the PBM, defined on an undirected graph $G(\mcal{V}, \mathbb{F}_2)$,
are considered, where the sample spaces of the variables are fixed to $\mcal{X}_i = \{-1,+1\}$.

The evaluations of $\ave{x_i}$ and $\ave{x_i x_j}$ for $\{i,j\} \in \mathbb{F}_2$ are essential for PBM learning.
From equation (\ref{eqn:k-SMCI}), these expectations are approximated by~\cite{Yasuda2015}
\begin{align}
m_{i}^{(1)}(\mathbb{S})&=\frac{1}{M}\sum_{\ell = 1}^M \tanh \gamma_i\big(\mbf{s}_{\mcal{N}_1(i)}^{(\ell)}\big)
\label{eqn:e1_1-SMCI}
\end{align}
and
\begin{align}
m_{i,j}^{(1)} (\mathbb{S})&=\frac{1}{M}\sum_{\ell = 1}^M \tanh\big[\atanh\big\{ \tanh\gamma_{i : j}\big(\mbf{s}_{\mcal{N}_1(i,j)}^{(\ell)}\big)
\tanh\gamma_{j : i}\big(\mbf{s}_{\mcal{N}_1(i,j)}^{(\ell)}\big)\big\} + w_{i,j}\big],
\label{eqn:e2_1-SMCI}
\end{align}
respectively, based on the 1-SMCI method, where
\begin{align}
\gamma_i(\mbf{s}_{\mcal{N}_1(i)}^{(\ell)})&:= w_i + \sum_{j \in \mcal{N}_1(i)}w_{i,j}\mrm{s}_j^{(\ell)},
\label{eqn:gamma_i}\\
\gamma_{i : j}(\mbf{s}_{\mcal{N}_1(i,j)}^{(\ell)})&:= \gamma_i(\mbf{s}_{\mcal{N}_1(i)}^{(\ell)}) - w_{i,j}\mrm{s}_j^{(\ell)},
\label{eqn:gamma_i-j}
\end{align}
and $\atanh$ is the inverse hyperbolic tangent function.
The detailed derivations of equations (\ref{eqn:e1_1-SMCI}) and (\ref{eqn:e2_1-SMCI}) are described in \ref{app:Derivation_1SMCI}.
Equations (\ref{eqn:e1_1-SMCI}) and (\ref{eqn:e2_1-SMCI}) are computable for any $G(\mcal{V}, \mathbb{F}_2)$.
However, the 2-SMCI methods for $\ave{x_i}$ and $\ave{x_i x_j}$ are not always computable;
for example, when $G(\mcal{V}, \mathbb{F}_2)$ is a dense graph such as $\mcal{R}_1(\mcal{T}) = O(n)$,
the computational cost of the evaluation of $m_{\mcal{T}}^{(2)}(\mathbb{S})$ is generally $O(2^n)$
because all the variables in $\mcal{R}_1(\mcal{T})$ must be summed over in the 2-SMCI method.

\subsection{Semi-second-order SMCI method}
\label{sec:s2-SMCI}

In the GSMCI method in equation (\ref{eqn:GSMCI}), the sum region $\mcal{A}$ can be freely selected.
The sum region should be as large as possible within the computational limitation.
For example, if $\mcal{A}$ is a (cactus) tree, equation (\ref{eqn:GSMCI}) can be computed using a generalized belief propagation~\cite{GBP2001};
if $\mcal{A}$ is a planar graph, it can be computed using a combinational technique~\cite{PlanarIsing2016}.
The appropriate choice of the sum region depends on the structure of $G(\mcal{V}, \mathbb{F}_2)$.

In the following, a setting of $\mcal{A}$ that is usable in general cases, semi-second-order SMCI (s2-SMCI) method, is proposed.
Consider a subset $\mcal{I}_1(\mcal{T}) \subseteq \mcal{N}_1(\mcal{T})$ in which there is no connected (or interacted) pair
i,e, any two different indices in $\mcal{I}_1(\mcal{T})$ belong to different cliques.
On the PBM, for $\mcal{A} = \mcal{T} \cup \mcal{I}_1(\mcal{T})$, the conditional distribution in equation (\ref{eqn:conditional-dist_GSMCI}) is represented as
\begin{align}
P\big(\bm{x}_{\mcal{A}} \mid \bm{x}_{\partial \mcal{A}} \big)
&\propto \exp\Big( \sum_{i \in \mcal{A}}\beta_i\big(\bm{x}_{\partial \mcal{A}}\big)x_i
+\sum_{\{i,j\} \in \{ c \in \mathbb{F}_2 \mid c \subseteq \mcal{T}\}}w_{i,j}x_ix_j \nn
\aleq
+ \sum_{i \in \mcal{T}}\sum_{j \in \mcal{I}_1(\mcal{T})}w_{i,j} x_ix_j\Big).
\label{eqn:conditional-dist_GSMCI_PBM}
\end{align}
where
\begin{align}
\beta_i\big(\bm{x}_{\partial \mcal{A}}\big)&:=w_i + \sum_{j \in \partial \mcal{A}}w_{i,j} x_j.
\label{eqn:beta_j}
\end{align}
In equations (\ref{eqn:conditional-dist_GSMCI_PBM}) and (\ref{eqn:beta_j}), $w_{i,j}$'s are regarded as zero when $\{i,j\} \not\in \mathbb{F}_2$.
The second term of the exponent of equation (\ref{eqn:conditional-dist_GSMCI_PBM}) denotes the interactions in the target region.
Because the variables in $\mcal{I}_1(\mcal{T})$ do not interact,
they can be analytically marginalized out from equation (\ref{eqn:conditional-dist_GSMCI_PBM}), leading to
\begin{align}
P\big(\bm{x}_{\mcal{T}} \mid \bm{x}_{\partial \mcal{A}} \big)
&=\sum_{\bm{x}_{\mcal{I}_1(\mcal{T})}}P\big(\bm{x}_{\mcal{A}} \mid \bm{x}_{\partial \mcal{A}} \big)\nn
&\propto
\exp\Big\{ \sum_{i \in \mcal{T}}\beta_i\big(\bm{x}_{\partial \mcal{A}}\big)x_i
+\sum_{\{i,j\} \in \{ c \in \mathbb{F}_2 \mid c \subseteq \mcal{T}\}}w_{i,j}x_ix_j \nn
\aleq
+\sum_{j \in \mcal{I}_1(\mcal{T})} \ln z_j\Big(\beta_j\big(\bm{x}_{\partial \mcal{A}}\big)  + \sum_{i \in \mcal{T}} w_{i,j}x_i \Big)\Big\},
\label{eqn:marginal_conditional-dist_GSMCI_PBM}
\end{align}
where $z_i(a) = \sum_{x_i \in \mcal{X}_i} \exp (a x_i) = 2\cosh a$.
Therefore, using the marginal distribution in equation (\ref{eqn:marginal_conditional-dist_GSMCI_PBM}),
the GSMCI method in equation (\ref{eqn:GSMCI}) can be reduced to
\begin{align}
m_{\mcal{T}}(\mcal{A}; \mathbb{S})=
\frac{1}{M}\sum_{\ell=1}^M \sum_{\bm{x}_{\mcal{T}}}f(\bm{x}_{\mcal{T}}) P\big(\bm{x}_{\mcal{T}} \mid \mbf{s}_{\partial \mcal{A}}^{(\ell)} \big),\quad
\mcal{A} = \mcal{T} \cup \mcal{I}_1(\mcal{T}).
\label{eqn:GSMCI_s2}
\end{align}
Equation (\ref{eqn:GSMCI_s2}) is the s2-SMCI method proposed in this section.
Equation (\ref{eqn:GSMCI_s2}) is computable in a dense graph (as long as the sum over $\bm{x}_{\mcal{T}}$ can be evaluated).
The s2-SMCI method is regarded as an intermediate approximation between the 1- and 2-SMCI methods
because it is identical to the 1-SMCI method when $\mcal{I}_1(\mcal{T}) = \emptyset$
and is identical to the 2-SMCI method when $\mcal{I}_1(\mcal{T}) = \mcal{N}_1(\mcal{T})$;
therefore, from the result obtained in theorem \ref{theo:GSMCI},
the approximation accuracy of the s2-SMCI method should be intermediate between those of the 1- and 2-SMCI methods.

\begin{figure}[tb]
\begin{center}
\includegraphics[height=3cm]{s2-SMCI.eps}
\end{center}
\caption{Illustration of the sum regions of the 1-, s2-, and 2-SMCI methods (the sum regions are shaded):
(a) the 1-SMCI method, (b) an example of the s2-SMCI method, and (c) the 2-SMCI method.
In the s2-SMCI method, another choice of the sum region is possible.}
\label{fig:s2-SMCI}
\end{figure}

Using equations (\ref{eqn:marginal_conditional-dist_GSMCI_PBM}) and (\ref{eqn:GSMCI_s2}),
the s2-SMCI methods for the expectations, $\ave{x_i}$ and $\ave{x_i x_j}$ ($\{i,j\} \in \mathbb{F}_2$),
are expressed as
\begin{align}
m_{i}(\mcal{A}; \mathbb{S})&=\frac{1}{M}\sum_{\ell=1}^M \tanh \xi_{i} \big(\mbf{s}_{\partial \mcal{A}}^{(\ell)} \big)
\label{eqn:e1_s2-SMCI}
\end{align}
and
\begin{align}
m_{i,j}(\mcal{A}; \mathbb{S})&=\frac{1}{M}\sum_{\ell=1}^M \tanh\big[\atanh \big\{ \tanh \xi_{i : j} \big(\mbf{s}_{\partial \mcal{A}}^{(\ell)} \big)
\tanh\xi_{j : i} \big(\mbf{s}_{\partial \mcal{A}}^{(\ell)} \big)\big\} + \omega_{i,j}\big(\mbf{s}_{\partial \mcal{A}}^{(\ell)} \big)\big],
\label{eqn:e2_s2-SMCI}
\end{align}
respectively, where
\begin{align}
\xi_{i}(\mbf{s}_{\partial \mcal{A}}^{(\ell)})&:=\beta_i\big(\mbf{s}_{\partial \mcal{A}}^{(\ell)}\big) + \sum_{j \in \mcal{I}_1(i)}\atanh \big\{ \tanh \beta_j\big(\mbf{s}_{\partial \mcal{A}}^{(\ell)}\big) \tanh w_{i,j}\big\},
\label{eqn:xi_i}\\
\xi_{i : j}(\mbf{s}_{\partial \mcal{A}}^{(\ell)})&:=\beta_i\big(\mbf{s}_{\partial \mcal{A}}^{(\ell)}\big)
+ \sum_{k \in \mcal{I}_1(i,j)}\atanh \big\{ \tanh \beta_k\big(\mbf{s}_{\partial \mcal{A}}^{(\ell)}\big) \tanh w_{i,k}\big\}\nn
\aldef
+\frac{1}{4} \sum_{k \in \mcal{I}_1(i,j)}\ln \frac{1 - \tanh^2\{\beta_k(\mbf{s}_{\partial \mcal{A}}^{(\ell)}) + w_{i,k}\} \tanh^2 w_{j,k}}
{1 - \tanh^2\{\beta_k(\mbf{s}_{\partial \mcal{A}}^{(\ell)}) - w_{i,k}\} \tanh^2 w_{j,k}},
\label{eqn:xi_i-j}\\
\omega_{i,j}\big(\mbf{s}_{\partial \mcal{A}}^{(\ell)} \big)&:=w_{i,j}
+ \sum_{k \in \mcal{I}_1(i,j)}\atanh\big\{ \tanh w_{i,k} \tanh w_{j,k}\big\}\nn
\aldef
+\frac{1}{4} \sum_{k \in \mcal{I}_1(i,j)}\ln \frac{1 - \tanh^2(w_{i,k} + w_{j,k}) \tanh^2 \beta_k(\mbf{s}_{\partial \mcal{A}}^{(\ell)})}
{1 - \tanh^2(w_{i,k} - w_{j,k}) \tanh^2 \beta_k(\mbf{s}_{\partial \mcal{A}}^{(\ell)})}.
\label{eqn:pmega_ij}
\end{align}
Equations (\ref{eqn:e1_s2-SMCI}) and (\ref{eqn:e2_s2-SMCI}) are computable in a dense graph.
In equations (\ref{eqn:xi_i})--(\ref{eqn:pmega_ij}), $w_{i,j}$'s are zero when $\{i,j\} \not\in \mathbb{F}_2$.
The detailed derivations of equations (\ref{eqn:e1_s2-SMCI}) and (\ref{eqn:e2_s2-SMCI}) are described in \ref{app:Derivation_s2SMCI}.

From the result obtained in theorem \ref{theo:GSMCI}, a larger $\mcal{I}_1(\mcal{T})$ is preferable.
However, the maximization of the size of $\mcal{I}_1(\mcal{T})$ is known as the \textit{maximum independent set} (MIS) problem, which is an NP-hard optimization problem.
The well-known greedy algorithm~\cite{MIS1997} for this problem is presented in Algorithm \ref{alg:greedy-MIS}.
In this algorithm, $\mrm{deg}(i;\mcal{U})$ denotes the degree of $i$ in the subgraph $\mcal{U}$,
and $\partial_{\mcal{U}}i:=\{j \mid j \in \mcal{U}, \> \{i,j\} \in \mathbb{F}_2\}$
denotes the set of the indices that are the first-nearest neighbors of $i$ in the subgraph $\mcal{U}$ i.e., $\mrm{deg}(i;\mcal{U}) = |\partial_{\mcal{U}}i|$.
\begin{algorithm}[H]
\caption{Greedy algorithm for MIS problem~\cite{MIS1997}}
\label{alg:greedy-MIS}
\begin{algorithmic}[1]
\State \textbf{Input} $\mcal{N}_1(\mcal{T}) \subseteq \mcal{V}$ and $\mathbb{F}_2$
\State $\mcal{I}_1(\mcal{T}) \gets \emptyset$ and $\mcal{U} \gets \mcal{N}_1(\mcal{T})$
\Repeat
\State Choose index $r$ such that $r = \argmin_{j \in \mcal{U}} \mrm{deg}(j;\mcal{U})$
\State $\mcal{I}_1(\mcal{T}) \gets \mcal{I}_1(\mcal{T}) \cup \{r\}$
\State $\mcal{U} \gets \mcal{U} \setminus \big(\partial_{\mcal{U}}r \cup \{r\} \big)$
\Until{$\mcal{U} \neq \emptyset$}
\State \textbf{Output} $\mcal{I}_1(\mcal{T})$.
\end{algorithmic}
\end{algorithm}

In step 4 in Algorithm \ref{alg:greedy-MIS}, one may encounter the case in which
multiple indices have the same minimum degree.
They are the equal candidates of $r$.
When the minimum degree is zero, the selection does not affect the final result
because all the candidates will be included in $\mcal{I}_1(\mcal{T})$.
However, when the minimum degree is larger than zero, one of them must be selected, and the selection can affect the final result.
A criterion for the selection is needed.
In this paper, a heuristic for the selection is proposed:
when multiple indices have the same minimum degree that is larger than zero,
the index that has the maximum $W_j$ is selected, where $W_j := \sum_{i \in \mcal{T} \cap \partial j}|w_{i,j}|$.
$W_j$ is regarded as the absolute strength of interaction between index $j$ and the target region.
This heuristic is based on our usual sense; that is, a pair having stronger interaction is more important.

\subsection{Experiment}

In this section, the validity of the proposed method is demonstrated using numerical experiments.
PBMs of $|\mcal{V}| = n= 20$ defined on two types of undirected graphs $G(\mcal{V}, \mathbb{F}_2)$ were used: a $4\times 5$ square grid graph and a random graph with connection probability $p$.
The bias and interaction parameters, $w_i$ and $w_{i,j}$, in the PBMs were generated from uniform distributions having the intervals $[-0.2,+0.2]$ and $[-0.3,+0.3]$, respectively.
On the PBMs, the approximation accuracies of the SMCI methods (the 1-, s2-, and 2-SMCI methods) were checked.
The accuracy of the approximation was measured by the mean absolute error (MAE) of the covariances:
\begin{align*}
\mrm{MAE}=\frac{1}{|\mathbb{F}_2|} \sum_{\{i,j\} \in \mathbb{F}_2}\big| \chi_{i,j}^{\mrm{exact}} - \chi_{i,j}^{\mrm{approx}}\big|,
\end{align*}
where $\chi_{i,j}^{\mrm{exact}}:= \ave{x_ix_j} - \ave{x_i}\ave{x_j}$ is the exact covariance, and $\chi_{i,j}^{\mrm{approx}}$ is its approximation.
Because the size of $\mcal{V}$ is not large, the exact covariance can be numerically evaluated.
The sample sets, $\mathbb{S}$, used in the SMCI methods were generated from the PBM by using Gibbs sampling based on a simulated annealing.

The plots in Figure \ref{fig:inf_result} show the results in (a) a $4\times 5$ square grid graph, (b) a random graph of $p = 0.2$,
and (c) a random graph of $p = 0.4$, for various $M$.
For the comparison, the results obtained by the standard MCI method are also plotted.
In the 2-SMCI method, the sum over $\bm{x}_{\mcal{R}_1(\mcal{T})}$ was numerically evaluated.
As expected, the SMCI methods are superior to the standard MCI method, and the accuracy of the s2-SMCI method is intermediate between those of the 1- and 2-SMCI methods.
The accuracies of the SMCI methods in (c) are worse than those in (a) and (b).
This indicates that they are more effective in a sparser graph.
In figures \ref{fig:inf_result}(a)--(c), the accuracy of the standard MCI method of $M = 1000$ is almost the same that of the 1-SMCI method of $M = 10$;
therefore, the standard MCI method needs an approximately 100 times larger sample set to reach the accuracy of the 1-SMCI method (at least in the settings of the present experiments).
The MAEs were also evaluated by using annealed importance sampling (AIS)~\cite{Neal2001}.
The results of AIS were almost the same as those of the standard MCI method.
In the AIS, the initial distribution was set to an uniform distribution over $\{-1,+1\}^n$,
and a sequence of the inverse temperature, $0 = \beta_0 < \beta_1 < \cdots < \beta_K = 1$, was set as $\beta_{k+1} = \beta_k + 10^{-4}$.

\begin{figure}[tb]
\begin{center}
\includegraphics[height=3.7cm]{inf_result.eps}
\end{center}
\caption{MAEs for $M = 10,100,1000$ in (a) a $4\times 5$ square grid graph, (b) a random graph of $p = 0.2$,
and (b) a random graph of $p = 0.4$.
The open and filled circles, ``$\circ$'' and ``$\bullet$'', denote the standard MCI and 1-SMCI methods, respectively;
the open and filled squares, ``{\tiny $\square$}'' and ``{\tiny $\blacksquare$}'', denote the s2- and 2-SMCI methods, respectively.
The plots are the average values over 200 experiments.}
\label{fig:inf_result}
\end{figure}

\section{Application to PBM Learning}
\label{sec:PBM_learning}

In this section, the learning of the PBM defined on $G(\mcal{V}, \mathbb{F}_2)$ is considered.
The PBM is represented with the explicit dependency on its parameters $P(\bm{x}) = P(\bm{x} \mid \theta)$,
where $\theta$ denotes the set of the parameters of the PBM, $\theta:= \{w_i ,w_{i,j} \mid i \in \mcal{V},\, \{i,j\} \in \mathbb{F}_2\}$.
Suppose that a training dataset consisting of $N$ training data points, $\mathbb{D}:=\{ \mbf{x}^{(\mu)} \mid \mu = 1,2,\ldots, N\}$, is obtained.
For the dataset, the log likelihood is defined as
\begin{align}
L_{\mathbb{D}}(\theta):=\frac{1}{N} \sum_{\mu = 1}^N \ln P\big(\mbf{x}^{(\mu)} \mid \theta \big).
\label{eqn:log-likelihood}
\end{align}
The log likelihood is the function with respect to $\theta$.
PBM learning is performed via maximum likelihood estimation (MLE),
that is, by maximizing the log likelihood with respect to $\theta$.
The parameters in $\theta$ are referred as to the learning parameters.
The gradients of the log likelihood with respect to $w_i$ and $w_{i,j}$, for $\mathbb{D}$, are
\begin{align}
g_i(\mathbb{D},\theta)&:= \frac{\partial L_{\mathbb{D}}(\theta)}{\partial w_i}=\frac{1}{N}\sum_{\mu = 1}^N \mrm{x}_i^{(\mu)} - \ave{x_i}, \\
g_{i,j}(\mathbb{D},\theta)&:=\frac{\partial L_{\mathbb{D}}(\theta)}{\partial w_{i,j}}=\frac{1}{N}\sum_{\mu = 1}^N \mrm{x}_i^{(\mu)}\mrm{x}_j^{(\mu)} - \ave{x_ix_j},
\end{align}
respectively. These gradients have the intractable expectations in their second terms.

An approximation based on the $k$-SMCI method was proposed~\cite{Yasuda2015},
in which the intractable expectations are approximated by the $k$-SMCI method i.e.,
$\ave{x_i} \approx m_i^{(k)}(\mathbb{D})$ and $\ave{x_ix_j} \approx m_{i,j}^{(k)}(\mathbb{D})$.
In this approximation, the sample set $\mathbb{S}$ is fixed to the dataset.
This approximation method based on the 1-SMCI method provides better learning results
than some known learning algorithms~\cite{SMCI2018}: MPLE~\cite{PL1975},
RM~\cite{RM2007}, and MPF~\cite{MPF2011}.

In the previous learning method~\cite{Yasuda2015}, the variables in the sample region were fixed by the given training set,
leading to an useful deterministic algorithm.
However, the accuracy of the learning cannot be improved without increasing the level of approximation of SMCI (i.e., increasing the value of $k$) in this method.
Moreover, the accuracy of the learning was degraded in a model-mismatched case~\cite{SMCI2018},
in which the graph structures of the data generating PBM and those of the learning PBM were different (or more precisely,
the graph of the learning PBM did not have that of the generating PBM as a subgraph).
This degradation was caused by fixing the sample set to the dataset.
An SMCI method is obtained by approximating the marginal distribution over the sample region by the sample distribution (cf. section \ref{sec:k-SMCI}).
In a model-mismatched case, the data distribution is no longer a good approximation of the corresponding marginal distribution in the sample region.

\subsection{Proposed learning algorithm}
\label{sec:proposed_learning}

The present paper proposes a new approximation for PBM learning, which is explained as follows.
At first, a sample set $\mathbb{S}$ with $M = eN$ is prepared, where $M$ is the size of the sample set, and $e$ is a positive integer called the ``data-extension rate.''
The sample set and the learning parameters are initialized as $\mathbb{S} = \mathbb{S}_0$ and $\theta = \theta_0$, respectively.
Using $\mathbb{S}_0$, the learning parameters $\theta_0$ are updated to $\theta_1$ based on a gradient ascent method with the approximate gradients
$g_i^{\mrm{app}}(\mathbb{S}_0, \theta_0)$ and $g_{i,j}^{\mrm{app}}(\mathbb{S}_0, \theta_0)$ defined by
\begin{align}
g_i^{\mrm{app}}(\mathbb{S}, \theta)&:= \frac{1}{N}\sum_{\mu = 1}^N \mrm{x}_i^{(\mu)} - m_i(\mcal{A}_i;\mathbb{S}), \\
g_{i,j}^{\mrm{app}}(\mathbb{S}, \theta)&:= \frac{1}{N}\sum_{\mu = 1}^N \mrm{x}_i^{(\mu)}\mrm{x}_j^{(\mu)} - m_{i,j}(\mcal{A}_{i,j};\mathbb{S}),
\end{align}
where $m_i(\mcal{A}_i;\mathbb{S})$ and $m_{i,j}(\mcal{A}_{i,j};\mathbb{S})$ are the approximations of $\ave{x_i}$ and $\ave{x_i x_j}$ on $P(\bm{x} \mid \theta)$,
respectively, based on the GSMCI method proposed in equation (\ref{eqn:GSMCI});
here, $\mcal{A}_i$ and $\mcal{A}_{i,j}$ are the sum regions determined for the corresponding target regions.
After the update of $\theta$, the sample set $\mathbb{S}_0$ is updated to $\mathbb{S}_1$ by using ($M$ parallel) $\kappa$-steps Gibbs sampling on $P(\bm{x} \mid \theta_1)$.
By using $\mathbb{S}_1$, the learning parameters are again updated to $\theta_2$ using the gradients $g_i^{\mrm{app}}(\mathbb{S}_1, \theta_1)$ and $g_{i,j}^{\mrm{app}}(\mathbb{S}_1, \theta_1)$.
This two-stages updating procedure, i.e., the parameter update and sample set update stages, is repeated during learning:
$\mathbb{S}_0, \theta_0 \to \theta_1 \to \mathbb{S}_1 \to \theta_2 \to \mathbb{S}_2 \to \cdots$.
The proposed procedure is inspired by the PCD method~\cite{PCD2008}.
The initial state of the sample set, i.e., $\mathbb{S}_0$, is set to the $e$-replicated $\mathbb{D}$.
The update procedure of the sample set is illustrated in figure \ref{fig:sample_procedure}.
The pseudocode of the proposed learning is presented in Algorithm \ref{alg:learning}.

\begin{figure}[tb]
\begin{center}
\includegraphics[height=5cm]{sample_procedure.eps}
\end{center}
\caption{Illustration of the update procedure of the sample set when $e = 3$. $\mathbb{S}_0$ is set to the $e$-replicated $\mathbb{D}$.
Each sample point in $\mathbb{S}_t$ is updated via $\kappa$-steps Gibbs sampling on $P(\bm{x} \mid \theta_{t+1})$;
therefore, $M (= eN)$ parallel Gibbs sampling is run to update the sample set.}
\label{fig:sample_procedure}
\end{figure}

\begin{algorithm}[H]
\caption{The proposed learning algorithm for PBM}
\label{alg:learning}
\begin{algorithmic}[1]
\State \textbf{Input} training dataset $\mathbb{D}$
\State Initialize the learning parameters: $\theta = \theta_0$
\State Initialize the sample set by using the $e$-replicated $\mathbb{D}$: $\mathbb{S} = \mathbb{S}_0$
\State Set $t = 0$
\Repeat
\State Update the learning parameters using a gradient ascent method with $g_i^{\mrm{app}}(\mathbb{S}_t, \theta_t)$ and $g_{i,j}^{\mrm{app}}(\mathbb{S}_t, \theta_t)$; e.g.,
\begin{align*}
w_i^{(t+1)} &\gets w_i^{(t)} + \varepsilon g_i^{\mrm{app}}(\mathbb{S}_t, \theta_t),\nn
w_{i,j}^{(t+1)} &\gets w_{i,j}^{(t)} + \varepsilon g_{i,j}^{\mrm{app}}(\mathbb{S}_t, \theta_t),
\end{align*}
where $\varepsilon$ is the learning rate.
\State Update the sample set $\mathbb{S}_{t}$ to $\mathbb{S}_{t+1}$ using ($M$ parallel) $\kappa$-steps Gibbs sampling on $P(\bm{x} \mid \theta_{t+1})$
(starting from $\mathbb{S}_{t}$)
\State $t \gets t + 1$
\Until{A certain criterion is satisfied}
\end{algorithmic}
\end{algorithm}

\subsection{Experiment}

In this section, the performance of the proposed learning algorithm described in the previous section is demonstrated.
In the following experiments, the training datasets of $N = 50$ were generated from a generative PBM (g-PBM) with $n = 20$,
which has the same form as equation (\ref{eqn:pairwiseMRF}), by using Gibbs sampling.
A training PBM (t-PBM) having the same size as g-PBM was trained using the generated artificial datasets.
Two cases are considered:
(i) the model-matched case in which g-PBM is defined on a $4 \times 5$ square grid graph and t-PBM is also defined on the same square grid graph;
and (ii) the model-mismatched case in which g-PBM is defined on a fully connected graph and t-PBM is defined on a $4 \times 5$ square grid graph.
Because the size of t-PBM is not very large, MLE can be performed exactly.
The parameters $w_i$ and $w_{i,j}$ in g-PBM were randomly selected according to uniform distributions having the intervals $[-0.2,+0.2]$ and $[-0.3,+0.3]$, respectively.
In the following experiments, $\kappa$ was fixed to one, and the learning rate was fixed to $0.02$.

The accuracy of the learning was measured by the MAE of the interactions:
\begin{align*}
\mrm{MAE}(t)=\frac{1}{|\mathbb{F}_2|} \sum_{\{i,j\} \in \mathbb{F}_2}\big| w_{i,j}^{(t)} - w_{i,j}^{\mrm{MLE}}\big|,
\end{align*}
where $w_{i,j}^{\mrm{MLE}}$ is the value obtained from the exact MLE, and $w_{i,j}^{(t)}$ is the trained value at step $t$.
Figures \ref{fig:train_result}(a) and (b) present the MAEs against the update step $t$;
(a) is the result of the (i) the model-matched case and (b) is that of (ii) the model-mismatched case.
In these figures, ``1-SMCI'' and ``s2-SMCI'' correspond to the training based on the 1-SMCI and s2-SMCI methods.
``(fix)'' means that the sample sets were fixed to the dataset ($\mathbb{S} = \mathbb{D}$), i.e., the learning strategy proposed in the previous studies~\cite{Yasuda2015,SMCI2018};
the others are the proposed method combined with the PCD-like strategy in which $e$ is the data-extension rate.
The proposed method greatly improved the accuracy in both cases.
The accuracies in the model-mismatched case were worse than those in the model-matched case.
However, the proposed method can reduce the accuracy degradation by increasing the value of $e$.

\begin{figure}[tb]
\begin{center}
\includegraphics[height=4.4cm]{train.eps}
\end{center}
\caption{
MAEs of interactions in (a) the model-matched case and (b) the model-mismatched case, versus the update step $t$.
The plots are the average values over 200 experiments.}
\label{fig:train_result}
\end{figure}

\section{Summary and Future Works}

In this paper, two different contributions for PBM were presented.
The first contribution is a generalization of the original SMCI method~\cite{Yasuda2015}, described in section \ref{sec:GSMCI}.
In the original SMCI method (i.e., the $k$-SMCI method), the setting of the sum region was seriously limited; i.e.,
for a target region $\mcal{T}$, the sum region must cover up to the $(k-1)$th-nearest-neighboring region, $\mcal{R}_{k-1}(\mcal{T})$, of the target region.
The statistical accuracy bound of the $k$-SMCI method was proved~\cite{Yasuda2015}(cf. Theorem \ref{theo:k-SMCI}).
However, a higher-order $k$-SMCI method cannot be applied in a dense graph
because the size of the sum region can be $O(n)$ there.
This study investigated a more flexible setting of the sum region and provided a statistical accuracy bound of the setting (cf. Theorem \ref{theo:GSMCI}).
The proposed method (i.e, the GSMCI method) allows a flexible setting of the sum region, such as in the s2-SMCI method discussed in section \ref{sec:s2-SMCI}.
The statistical accuracy bounds of the $k$-SMCI and GSMCI methods were validated in generalized MRFs.

The second contribution of this study is a new algorithm for PBM learning, described in section \ref{sec:PBM_learning}.
The proposed learning method greatly improved the accuracies of learning in the model-matched and model-mismatched cases.
The learning method proposed by the previous study~\cite{Yasuda2015,SMCI2018} is applicable to only fully visible PBMs,
because the values of the variables in the sample region must be filled by the dataset.
The proposed learning method can be immediately applied to PBM learning with hidden variables,
such as RBMs (and its variants: Gaussian-Bernoulli RBMs~\cite{GBRBM2011} and Gaussian-Spherical RBMs~\cite{GSRBM2020}) and DBMs,
because the values of the (hidden) variables in the sample region are filled by the sample points obtained by Gibbs sampling.
Application to RBMs and DBMs is considered as important future works.

SMCI uses a sample set drawn from an MRF; therefore, the approximation accuracy of SMCI depends on the quality of the sampling algorithm.
In the experiments of this paper, usual Gibbs sampling was used.
An effective sampling method, based on belief propagation, was proposed~\cite{BP+MCMC2014}, the ideas of which is close to SMCI.
It is believed that a combination of both SMCI and this sampling method leads to more effective approximations.
This is also considered as an important future work.

\subsection*{Acknowledgment}
This work was partially supported by JSPS KAKENHI (Grant Numbers 15H03699, 18K11459, and 18H03303),
JST CREST (Grant Number JPMJCR1402), and the COI Program from the JST (Grant Number JPMJCE1312).

redpajama_set_name: RedPajamaArXiv


text:
Q: PGFPLOTS: How To Put \addplot boxplot prepared Inside Macro Definition I have a need to wrap the `\addplot [boxplot prepared]' inside a macro to greatly save typing. It seems simple enough, but the devil is in the details with macro expansion issues. Is this doable given PGFPLOTS ? I looked through the Programming Notes as well as the PGFPLOTS 1.11 Manual, with nothing coming up. I tried dumping the macro to the log file with the \meaning command. It gives me a 'Invalid character' message for every character in the macro. Not much use there. Any insights would be greatly appreciated. Its frustrating something so fundamentally simple would bring a project to a dead stop. My MWE is below.
\documentclass[border=3mm]{standalone}
\usepackage{pgfplots}
\pgfplotsset{compat=1.11}
\usepgfplotslibrary{statistics}

\newcommand{\desboxplot}[6]{
 \addplot[ boxplot prepared={
  lower whisker={#1}, lower quartile={#2}, median={#3},
  upper quartile={#4}, upper whisker={#5}, average={#6},
  box extend=0.5,  % height of box
  whisker extend=0.5, % height of whiskers
  every box/.style={thin,draw=black,fill=gray!50},
  every whisker/.style={black,thick},
  every median/.style={black,thick},
  every average/.style={draw=red, /tikz/mark=* },
  /pgf/number format/precision=2 } ]
  coordinates {}
  node[left,black] at
  (boxplot box cs: \boxplotvalue{lower whisker},0.5)
  {\tiny\pgfmathprintnumber{\boxplotvalue{lower whisker}}}
  node[right,black] at
  (boxplot box cs: \boxplotvalue{upper whisker},0.5)
  {\tiny\pgfmathprintnumber{\boxplotvalue{upper whisker}}};
} %end of \desboxplot definition

\begin{document}
\begin{tikzpicture}
\begin{axis}[
y=1.5cm,
]
\addplot+[
  boxplot prepared={
    lower whisker=5,
    lower quartile=7,
    median=8.5,
    upper quartile=9.5,
    upper whisker=10,
    box extend=2,  % height of box
    whisker extend=2.2, % height of whiskers
    every box/.style={very thick,dashed,draw=black,fill=yellow},
    every whisker/.style={red,ultra thick},
    every median/.style={densely dotted,cyan,ultra thick},
  },
]
table[row sep=\\,y index=0] {
data\\ 1\\ 3\\
};
\meaning\desboxplot %{5}{7}{8.5}{9.5}{10}{9}
\end{axis}
\end{tikzpicture}
\end{document}

A: You need to add a % at the end of the \addplot[ boxplot prepared={ line.
\documentclass[border=3mm]{standalone}
\usepackage{pgfplots}
\usepgfplotslibrary{statistics}

\newcommand{\desboxplot}[6]{%
 \addplot[ boxplot prepared={%
  lower whisker={#1}, lower quartile={#2}, median={#3},
  upper quartile={#4}, upper whisker={#5}, average={#6},
  box extend=0.5,  % height of box
  whisker extend=0.5, % height of whiskers
  every box/.style={thin,draw=black,fill=gray!50},
  every whisker/.style={black,thick},
  every median/.style={black,thick},
  every average/.style={draw=red, /tikz/mark=* },
  /pgf/number format/precision=2 } ]
  coordinates {}
  node[left,black] at
  (boxplot box cs: \boxplotvalue{lower whisker},0.5)
  {\tiny\pgfmathprintnumber{\boxplotvalue{lower whisker}}}
  node[right,black] at
  (boxplot box cs: \boxplotvalue{upper whisker},0.5)
  {\tiny\pgfmathprintnumber{\boxplotvalue{upper whisker}}};
} %end of \desboxplot definition

\begin{document}
\begin{tikzpicture}
\begin{axis}[
y=1.5cm,
]

\desboxplot{5}{7}{8.5}{9.5}{10}{9}
\end{axis}
\end{tikzpicture}
\end{document}

redpajama_set_name: RedPajamaStackExchange


text:
On the Prairie Diamond: The Weblog of LeAnne Howe
About Howe's Books
LeAnne Howe
Some Things I've Done, Places I've Been
Awards, New Books, New Friends
by LeAnne Howe
Congratulations Tyehimba Jess for winning the 2017 Pulitzer Prize for Poetry for Olio. I first met Tyehimba in 2005 at the University of Illinois, Urbana-Champaign. That year Illinois hired author Rigoberto Gonzales, (Butterfly Boy, among a dozen more books), Tyehimba Jess (Leadbelly, Oilo) and me (Shell Shaker, Miko Kings: An Indian Baseball Story). This last year I taught Olio in my graduate symposium in English at UGA Athens. What a fascinating read. Rush out and buy it. Read it. Chew on the pages.
Just returned from the NonFictionNow Conference in Reykjavik, Iceland. Below is a powerful waterfall, not from from the capital city. The countryside in Iceland is gorgeous, in some places other worldly, lots of volcanos simmering, waiting to erupt. And the conference had all the rockstars of CNF, myself excluded.
I'm rushing off tomorrow to London, UK for the American Indian Workshop (AIW) hosted this year by friends and colleagues David Stirrup and Padraig Kirwan. Stay tuned for lots of pictures and news from the conference. Lots of American Indian and First Nation friends will be there. Very excited to see everyone.
I recently co-authored a chapbook of poems with Irish poet Doireann Ní Ghríofa titled, Singing, Still, Libretto for the 1847 Choctaw Gift to the Irish for Famine Relief. The chapbook is a series of "call and response" poems dedicated to our ancestors, the Irish and the Choctaws who lived and died through the hunger years, suffering at the hands of colonialism. The chapbook poetry is trilingual, allowing English to form a bridge between our native languages. Here's a link to our story that ran in the Irish Times. https://www.irishtimes.com/culture/books/famine-bonds-choctaw-and-irish-poets-combine-1.3130918
Friends, Readings, New Books, Conferences
These days part of the job of an academic and writer is to host readings by great writers. I know what you're thinking: that's not a hard job. Indeed it isn't.
On January 24, I hosted University of San Francisco poet and scholar Dean Rader (Okie) to UGA's campus. He was funded by the Eidson Foundation at UGA that also funds my chair. Dean gave a wonderful reading from his new book at Cine in Athens, GA. His new book Self Portrait as Wikipedia Entry as https://www.coppercanyonpress.org/pages/browse/book.asp?bg=%7B603C6409-E9AF-4F8F-B6BF-8BCECC32754A%7D is a great read and UGA was proud to be his premier reading for the new book just out from Copper Canyon Press.
Also in January 2017, the Georgia Review hosted a reading by poet Jericho Brown (Emory University) on the UGA campus. He read from some of his new work. Both poets gave fantastic readings. UGA audiences live for poetry readings.
Last summer I was fortunate to have worked with Jericho Brown and Native poet Natalie Diaz at the Minnesota Northwoods Writers Conference in Bemidji held June 20-26 2016 at the American Indian Resource Center on the campus of Bemidji State University. Pictured below, left to right, are Brenda Child, me, Natalie, and Jericho at a local pizza restaurant in Bemidji. Brenda Child (Red Lake Ojibwe) is a historian at the University of Minnesota, and author of Boarding School Seasons, and most recently My Grandfather's Knocking Sticks. https://indiancountrymedianetwork.com/history/events/growing-up-ojibwe-brenda-j-child-wins-award-for-historical-family-memoir/. In 2003, we were in Bilbao, Spain for a conference and she was consistently the best judge of Spanish red wines I've ever had the privilege of hanging out with! Brenda and I met in the Twin Cities years ago when I had a visiting writer position at Carleton College, Northfield, MN. And speaking of the Twin Cities, this year the Native American Literature Symposium (NALS) was held at Mystic Lake Casino, just outside of Minneapolis, February 2-4, 2017. This was our 18th year to host NALS somewhere on Native land in the United States. I'm proud to have hosted Payton Guthrie, (Choctaw) filmmaker from the Choctaw Nation of Oklahoma, Durant. He gave the Friday night's keynote lecture at NALS.
Keynote Payton Guthrie (Choctaw). His address was supported in part by the Eidson Foundation Fund, and the Choctaw Nation of Oklahoma, and NALS.
Some NALS photos from the conference below.
Below is a picture of my recent lecture at the University of Oklahoma, Norman, and it was my great pleasure to be home again. Will see what happens next!
"Quote The Raven Nevermore"
"But the Raven still beguiling all my fancy into smiling,
Fancy unto fancy, thinking what this ominous bird of yore —
What this grim, ungainly, ghastly, gaunt, and ominous bird of yore
Meant in croaking, Nevermore." – Edgar Allen Poe, 1845
Why a picture of Queen Elizabeth's raven near the Tower of London and the quote from Edgar Allen Poe? Well, we just had the warmest Halloween ever in the US and I was longing for the cold. But the truth is the picture was taken on June 1, 2016 in the coldest winter I ever spent last summer in London, to misquote Mark Twain. Honestly last June, the temperatures were in the 40s with gail winds most of the week we were in the Queen's city. We had traveled from Atlanta to Denmark, and onto London.
Short non sequitur. While in London, Padraig Kirwan and I gave a public presentation at Goldsmiths University of London on The Trans-Atlantic Choctaw Irish-Exchange 1847-1995, a book of essays by Choctaw scholars and Irish scholars about the gift from the Choctaws (money) in 1847 to the Irish during the terrible potato famine in that country.
Here we are in London
Talking briefly before our lecture Goldsmiths University on June 1, 2016, Padraig Kirwan, (front right) we chatted with friends and colleagues. (Back left, western writer Emry McAlear and his wife, Brigitte McAlear now living in London came to the lecture.) Emry is an alum at Vermont College of Fine Arts. I was fortunate to work with him on his collection of essays, Townie Boy, at VCFA. His award-winning essays on life in Montana are both poignant and funny.
Back to the Tower of London.
I took the photo at the Tower of London complex where legend has it that six ravens must guard the Tower of London at all times or the monarchy will fall. (I'm quoting from the Tower's PR.) We walked around and took a tour. Crimony, throughout history over 3,000 people have had their heads removed off at the Tower of London, including Anne Boleyn, former wife of Henry VIII. (And they called Indians savages.) They killed her in mid-prayer, so said the Beefeater giving us the tour. He said her mouth kept moving for 28 seconds after they lopped off her head. Who had the terrible job was counting the seconds?
View of the Thames River from the Tower site.
As we walked central London, near the Foundling Home and Charles Dickens House and museum site, we passed a school that raises goats. Kinda awesome when you think this is central London.
The goats Of London. Not far from the Halingford Hotel, a great place to stay when in London, is a schoolyard of goats. For milk I presume. But in all the hustle and rumble of central London this two-acre urban farm is a welcome respite.
A week before the June trip to London we'd traveled to Copenhagen, Denmark to give papers at the 37th Annual American Indian Workshop held in Odense, Denmark. The theme was Indian humor. . . Aside: there was a dust up among European colleagues about how one knows Indians are funny. Little Big Horn massacre leaps to mind. . . We reconnected with friend and historian Susan Sleep Smith, Michigan State University, her paper on Indian and French humor was fantastic. She and I met at a 1995 NEH summer seminar at the University of Oklahoma. That summer is when I met historian Jacki Rand (Choctaw) and we've stayed in touch all these years later. She's one of the scholars writing for a chapter for The Trans-Atlantic Choctaw Irish-Exchange 1847-1995 that Padraig and I are editing. In fact Jacki's house in Ada, Oklahoma is right across the street from mine.
Moving at a snail's pace on this summer 2017 travelogue, (ah hem) while in Odense, Denmark, we visited the Museum of Hans Christian Andersen. Below. His work has been translated into 125 different languages and the copies of the breadth of these translations are in the Odense museum. It was impressive to see all the texts translations.
Jim Wilson is standing far right from the entrance to Hans Christian Andersen Museum in Odense.
Before Denmark and before London, we'd traveled this summer to the Native American and Indigenous Studies (NAISA) conference in Honolulu, Hawaii. Jim Wilson and I gave papers (he did) and I chaired a panel at the conference. Pictured here with a dear friend Professor Noenoe Silva (Native Hawaiian) whose seminal work, Aloha Betrayed, I teach from. She has a new book coming out soon.
While at NAISA, I got to see, and at least hug one of my graduate students from the University of Illinois, Eman Ghanayem, and Dr. Theresa Rocha. Congrats Theresa on completing your PhD this past year. What a treat to see you both. Coming next: the future books I want to read. Aren't you glad the US Presidential Election is almost over!
Travels and New Paths On the Horizon
Over the past five months I've traveled a great deal, made new friends, connected with old friends, and read a few new books, even given readings from my new manuscript. This blog post is perhaps more of a diary entry because it helps me remember where I've been. And where I'm going.
On March 2, 2016 I traveled to Brigham Young University in Provo, Utah for the Branding the American West Symposium, and to celebrate the publication of the book (above) by the same name. My friend, poet and scholar Dean Rader got me involved in the book project and art exhibition about two years ago. We all had various assignments, his was to write about art and landscapes. His essay, "Part of the Strangeness, Notes On Landscapes And the American West," tackles artists such as Walter Ufer, Maynard Dixon, and Kenneth Miller Adams, among others, and discuss the tensions in their work. My assignment was to write about classic Hollywood movies and representations about Indians. "Imagine There's No Cowboy, It's Easy If you Try," is about images in films that portray American Indians as "savages." It's a first person account of growing up in Oklahoma and watching bad Westerns on tv every Friday night and hoping that just once the Indians would defeat the cowboys. It never happened. Today the academic industry of critical inquiry known as "settler colonialism," frankly does nothing to alleviate the suffering of "the person[s] to whom things happen," paraphrasing Virginia Woolf. Uh-hem.
In Branding The American West, Rader's essay opens the book and mine closes it. Call us bookends. The lovely coffee table book, edited by Marian Wardle and Sarah Boehme, is filled with essays about the west. Authors included are: Jon Ott, Jimmy L. Bryan, Susan Rugh, and Elizabeth Hutchins. Check it out at your local bookstores or online.
Minerva K. Teichert's Stampede in the Canyon.
I love Teichert's painting, it's large and very romantic and I think of it as a long shot in the 1950s tv show, The Lone Ranger. I also wrote about The Wind. Crimony, it's one of the scariest silent films ever. The Wind is a good story and Lillian Gish is a remarkable actress. Her gestures in the silents are always on the mark, powerfully acted. But sad to say that in The Wind, American Indians are portrayed as menacing demons. Typical.
From the lectern, "Oh lord, give me strength these films are dreadful. "
Pictured below are some of the students from Patrick Madden's (far left) BYU creative writing class that I got to meet while at the symposium. They asked great questions about the writing process, and how a writer chooses the stories he or she chooses to tell.
Some of the faculty from the USA and UK that attended the Branding The American West Symposium March 2016. Far right is Wendy Castenell, Assistant Professor of African American Art, University of Alabama.
Searching for Sequoyah, A New Documentary Film in the Making
Beginning on March 9, 2016 we began filming interviews from our new documentary Searching For Sequoyah. The film is about the Cherokee genius Sequoyah who in 1808 began working on a system to write the Cherokee language. He worked in secret. Some people thought he was crazy. Others thought he must be practicing sorcery, threading sounds on an invisible symbol. Finally, in 1821, Sequoyah perfected his syllabary, a symbol set with one character for each syllable in the language—86 of them. It is the only indigenous writing system north of Meso-America.
Searching for Sequoyah's co-producer and cinematographer James M. Fortier and myself, co-producer and screenwriter, are working along with Dr. Jace Weaver, Director of UGA's Institute for Native American Studies, and University of Oklahoma professor Dr. Joshua Nelson (Cherokee) on the new documentary. We interviewed Sequoyah's lineal descendants in Tahlequah, OK in March along with Dr. Daniel Justice, (Cherokee) who traveled from University of British Columbia, Vancouver for the film project. We are very grateful that he came for the filming. We will will likely continue to work on the project for the next couple of years. Perhaps longer. Pictured left to right. Winnie Guess Perdue, Joshua Nelson, and John Ross. Both Perdue and Ross are direct descendants of Sequoyah.
Left to right: Joshua Nelson, Grace Milley, and Jace Weaver on the grounds at the Cherokee Museum. March 2016.
James Fortier filming in downtown Tahlequah.
Left to right: Joshua Nelson, Grace Milley, Daniel Justice, (me) and Matt Jennings dining after a day of filming in Tahlequah.
March 17-20, 2016 Native American Literature Symposium, NALS, Albuquerque, New Mexico.
The highpoint of this year's Native American Literature Symposium (NALS) 2016 was a children's dance they performed for the conference goers. This is our 16th year for hosting NALS.
Pictured left to right are Miriam Brown Spiers, Padraig Kirwan, and myself, NALS 2016. Padraig and I are editing a collection of essays, "Transatlantic Reciprocity: Contextualizing Exchanges between the Choctaw & the Irish. "
NALS academics 2016, Albuquerque. Gwen Westerman, left, NALS symposium director. Craig Howe, keynote speaker at NALS, 2016, far right, and creator of CAIRINS in South Dakota.
Indians sharing a dirty joke. NALS 2016. (I don't know why I say such things. Everyone knows Indians don't tell dirty jokes and we don't laugh. Check out a classic Hollywood western, Indians are not laughing.)
More NALS academics 2016. All friends and wonderful scholars.
ABQ, New Mexico was fun, but it's time to move onto AWP conference in Los Angeles, CA March 30-April 2, 2o16.
UGA's Creative Writing Program hosted an off-site reading during the 2016 AWP conference in LA. We called our event, "shock and awe." (Kidding.) Pictured here are UGA's very excellent graduate students. Pictured here with the Vermont College of Fine Arts 35th Anniversary Reading group. We all read from our new work. Left to right: Bret Lott, Neela Vaswani, Alison Deming, Rigoberto Gonzalez, me, and Ellen Lesser. Here we are at the book signing for Cutthroat Journal's beautiful new anthology at AWP 2016.
AWP 2016. LA has some cool restaurants. See the blue flame below.
"After the absinthe." See the blue fire in the glass in the picture above, 2016. Left to right. Chris, (me), Dean Rader, Jonathan Silverman.
Hot Off the Press: Red Ink
Red Ink. Spring 2016. Issue 18.1 contains poetry, fiction and drama, an interview with Sherman Alexie.
That takes me up to my visit at Pembroke NC, and my visit to University of North Florida. I"ll save that for another post. Yeah, like I said, this is mostly a diary of what I've been doing the last eight weeks or so. It's been a busy semester, but I've managed to read two great new books, Olio O by Tyehimbia Jess, and handholding: 5 kinds by Tracie Morris. Check'em out. As for me I've completed a new book manuscript. Will see what happens next.
What I'm Reading And Re-Reading in 2016
From "Our Indigenous Flesh" a reading and discussion event sponsored by our press, Aunt Lute Books on September 24, 2015, Galeria de la Raza in San Francisco, CA.
I'm re-reading Ire'ne lara Silva's Flesh to Bone, (Aunt Lute Books 2013). A brilliant collection of nine stories of myth and fire and ultimately transformation. The prose is stunning.
Other books I'm reading are Ed Pavlic's Who Can Afford to Improvise? James Baldwin and Black Music, the Lyric and the Listeners. A great read. My favorite chapter is "Shades Cannot Be Fixed, On Privilege, Blindness, and Second Sight."
For a change of pace check out Paris Was Ours by Penelope Rowlands. She's gathered writers and essayists such as C.K. Williams, Simon Shimon, David Sedaris, Julie Lacoste and many others who talk about their lives since moving to Paris. The essays made me wish I'd emigrated to Paris about fifteen years ago. Alas. I know, I know Paris has been badly abused in 2015, but the Parisians refused to be broken in spirit by terrorists. Good for them. I would love to move there for a year, or ten, but that will have to wait for a while until I can save a ton of money.
Patrick, left, Brian right, the Heirloom Cafe, New Year's Eve 2015.
Since moving to Athens, Georgia in August 2014, I've made a good habit of dining at the Heirloom Cafe on Chase Street. This New Year's Eve I took a break from reading and writing and had another incredible meal at Heirloom. Four courses paired with delicious wines. And we began with champagne. A great part of the charm of Heirloom are the baristas, chefs, and sommelier. Picture above are Patrick (sommelier) and Brian, (he makes the best martinis in the entire universe) two foodies and fab hosts, and friends, at the Heirloom. I highly recommend the restaurant. It's my home away from home.
At the Heirloom Cafe, New Year's Eve. Cheers!
After eating and drinking my way to New Year's Day, I sat on the couch and re-read Dr. Tatiana's Sex Advice To All Creation by Olivia Judson. It's written from the point of view of moorhens, stick insects, fig wasps and many other creatures great and small all asking for sex advice. It's smart, funny and just the kind of book you will want to read after too many glasses of bubbly. This from page 69.
Dear Dr. Tatiana,
My name's Rob, and Im a bedbug. Xylocoris maculipennis. I've read that if I have sex with my friend Fergus, he'll deliver my sperm when he next has sex with Samantha. Is this for real?
Making Mischief between the Sheets
See what I mean. Something for everyone. I tend to read this book about every 2 years since it first came out.
Also on my bedside table is a copy of Jimmy Carter's Our Endangered Values: American's Moral Crisis. After watching my fill of this year's Presidential debates and how the candidates snipe at each other, I recommend reading instead of watching television. The Presidential election has become a race of proliferating nonsense. See Carter's chapter "Attacking Terrorism, Not Human Rights."
As for me it's back to teaching beginning January 11, 2016. My new course is The Walking (un)Dead: American Indians as Zombies, Ghosts, and Ghouls in 20th century novels. (Again, something for everyone.) I've long thought American Indians are the original zombies at least as far as stereotypes go. In my future blog posts in coming months I'll be posting about how the course is going. And other events.
Etowah Mounds: The Place To Be One Thousand Years Ago!
by ekt
from → Choctalking, Indians at Play
Spring At Last!
Georgia! Thank goodness the cranky weather, cloudy, rainy holidays like St. Patrick's Day, freaky ice storms — are behind us now. Just in the nick of time. March has been a bit of a knicker-pincher in terms of the events I've hosted beginning on March 5, 2015 with Lo! The Poor Indian No More: The Trans-Atlantic Choctaw Irish Exchange 1848-1995, A Colloquium at the Russell Special Collections Library at the University of Georgia.
In 1847, the Choctaw Indians at Skullyville, Indian Territory were saddened to hear the news of the starvation in Ireland due to the potato famine. The Choctaw had experienced starvation only sixteen years earlier, when the entire Choctaw Nation of people were forced to walk west by Andrew Jackson's government. The event is known as the Trail of Tears. Choctaws were the first to be "removed" out of Southeast and their ancient homelands. The Nation lost one quarter of its population. Fast-forward to Skullyville, Indian Territory. The Choctaws collected $170.00 (or $710.00, there's a controversy about the money) and forwarded the funds to the U.S. famine relief organization.
Paul Gleeson, Consul General of Ireland, Atlanta began our event on March, 5, 4:30 p.m., at the University of Georgia and told us that incredible as it sounds, Ireland's population has never recovered from the famine disaster of 1847-1848. Before the famine Ireland's population was eight million. 168 years later the population of Ireland is barely six million.
The panelists for the colloquium were Jacki Rand, (citizen, Choctaw Nation of Oklahoma), associate professor of history at the University of Iowa; Padraig Kirwan, Senior Lecturer in the Department of English and Comparative Literature, at Goldsmiths, University of London, UK; Peter D. O'Neill, Assistant professor in the Comparative Literature Department at UGA; Jace Weaver, Franklin Professor of Native American Studies and the Director of the Institute of Native American Studies at UGA, and me. We had a marvelous crowd of about 90 people, a mix of the public, students and faculty and I'm so grateful for the support everyone has shown me. The March 5 event was to kick off our book-length project on the history of the Irish Choctaw exchange. Pictured below, left to right, Padraig Kirwan, Peter O'Neill, Paul Gleeson, Jace Weaver, Jacki Rand, and me.
Following the Choctaw Irish Colloquium, the UGA Creative Writing faculty flew to Missoula, Montana for "Thinking Its Presence: The Racial Imagery" March 12-14, 2015, a conference on race in the 21st century. The panel reading was Thursday afternoon and organized by poet and fiction writer, Maggie Z. Her new book Companion Animal is coming out in a few hours, (actually April 1) congratulations Maggie. See back and front below. Cool, beans, huh!
Pictured below are the UGA creative writers, left to right, Magdalena Zurawski, Gabriel Fuentas, Shamala Gallegher, and Ed Pavlic. The next day I had to leave Missoula, Montana and fly to Albuquerque, New Mexico for the Native American Literature Symposium (NALS), March 12-14, 2015. I've been part of the organizing group that's hosted NALS for the past 16 years. We've always held the conference on Native land, another important aspect of the conference.
Pictured below are NALS regulars at the Isleta Casino where the conference was held this year. I can hardly believe it, that 16 years have passed so quickly. I'm very proud of all the literary scholars that have come through NALS over the years.
My last event to host in March at UGA is a reading by Susan Power, author of the 2013,
Sacred Wilderness, Roofwalker, and The Grass Dancer. Power's reading, March 26, 2015 will be at 4:30 p.m. at the North Auditorium on the UGA campus. Power is also reading and signing books at Avid Books in Athens, GA from 6:30-7:30 p.m.
Next, the UGA CW faculty will be reading at Cine Theater on Hancock in Athens, GA March 31, 2015 from our work. 7 p.m. (I'll be reading new poems.)
On April 4, 9:30 p.m. I'll be reading at the Scissortail Creative Writing Festival, East Central University, Ada, Oklahoma. YAH!
About LeAnne Howe
LeAnne Howe is an enrolled citizen of the Choctaw Nation of Oklahoma. She writes fiction, poetry, screenplays, creative non-fiction, plays and scholarship that primarily deal with American Indian and Native American experiences.
Her first novel Shell Shaker (Aunt Lute Books, 2001) received an American Book Award in 2002 from the Before Columbus Foundation. The novel was a finalist for the 2003 Oklahoma Book Award, and awarded Wordcraft Circle Writer of the Year, 2002. Equinoxes Rouge, the French translation, was the 2004 finalist for Prix Medici Estranger, one of France's top literary awards. Evidence of Red (Salt Publishing, UK, 2005) won the Oklahoma Book Award for poetry in 2006, and the Wordcraft Circle Award for 2006. Her most recent novel is Miko Kings: An Indian Baseball Story (Aunt Lute Books, 2007). Her latest two books Choctalking On Other Realities (Aunt Lute Books), a memoir, and Seeing Red Pixeled Skins, American Indians and Film (Michigan State University Press), a co-edited anthology of film reviews were both published in 2013. She is the Eidson Distinguished Professor of American Literature in the English Department at the University of Georgia, Athens. She is available for readings and lectures at colleges and universities.
Contact: ileannehowe at gmail dot com
The Official Biography
Recent Twitterings
futurism.com/neoscope/stran… 1 week ago
@MichelleObama @GretaThunberg Go Greta. You teach us all! 1 month ago
History in the making, President Trump must be impeached or resign. He's tried to drag Ukraine to cheat in order t… twitter.com/i/web/status/1… 1 month ago
Impeach Donald J. Trump. Based on the facts and merits of the case against him, we should impeach him. 1 month ago
@MiaFarrow @realDonaldTrump Ditto. We all feel embarrassed by this president 3 months ago
hollrahp on Travels and New Paths On the Horizon
Abdul on Spacetime, Arabizi, and Metaphor over Amman
cristinarojo on In Passing: Greg Rodgers, 1968-2014
ekt on Happy Halloween 2014! Athens, Georgia
Patrice Hollrah on Happy Halloween 2014! Athens, Georgia
American Indian Literature and Arts
Jarryd Lowder
Native American Literature Symposium
Newspaper Rock
Santee Frazier
Storytellers: Native American Authors Online
Then There's None
Baseball History Resources
American Indian Baseball Players
Ron Kaplan's Baseball Bookshelf
The Statistics

redpajama_set_name: RedPajamaCommonCrawl


Krosnick, Jon A., Neil Malhotra, Cecilia Hyunjung Mo, Eduardo F. Bruera, LinChiat Chang, Josh Pasek, and Randall K. Thomas. 2017. "Perceptions of Health Risks of Cigarette Smoking: A New Measure Reveals Widespread Misunderstanding." PLOS ONE. 12(8): e0182063.
Malhotra, Neil. 2015. "An Empirical Analysis of `Tort Tales`: How Cultural Memes Influence Attitudes on Tort Reform." Journal of Law and Courts. 3(1): 149-166.
Malhotra, Neil, and Edmund Helmer. 2012. "Inflation in Weekend Box Office Estimates." Applied Economics Letters. 19(14): 1411-1415.
Malhotra, Neil, and Matthew E. Carnes. 2008. "Political Stability under Uncertainty: Applying Bounded Rationality to the Study of Governance and Civil Conflict." British Journal of Political Science. 38(1): 45-64.


text:
COMMITTED TO PRODUCTS THAT ARE-NATURALLY BEAUTIFUL.
From the lap of the Rocky Mountains comes a group of environmentally conscious professionals who nurture a profound appreciation of Nature and its wonders. Dryleaf is a community of young entrepreneurs with diverse origins who share a common belief in ensuring that our environment is sustained for the generations to come after us. We all believe that we as human can do better in finding more creative and environmentally viable ways to maintain a healthy lifestyle in today's modern world.
We at Dryleaf are committed to solving this problem by introducing an eco-friendly, biodegradable and affordable solution to dinnerware plates for homes, businesses and restaurants. Our solution came from an ancient practice that has been in use for more than 1000 years. Areca leaf plates and bowls are made from disposable Areca Palm leaf leaves that come from a 100% natural source. No artificial material is added and they can be made into different shapes and forms according to the purpose intended.
From ancient India comes the inspiration for Dryleaf's solution to environmental pollution. Our disposable, environmentally-friendly dinnerware and dining accessories are designed especially for today's audience in mind. Affordable, health-friendly and naturally insulating, Dryleaf Dinnerware are great for your picnics, dinners, receptions and even everyday use at home for an eco-friendly household. Our products are also microwave safe, practical and very trendy.We are a socially responsible company that is dedicated to helping the environment by making naturally occurring and degrading dinnerware of extremely high quality.
We hope that we can influence people to use more environmentally-friendly options in their daily life. We also partner with NGOs and agencies that are committed to raising awareness about the dangers of plastics and we support training programs aimed at supporting and helping families.


text:
Who the F#ck Is Chip Seinfeld? (2011)
Wendy Liebman: Taller on TV (2011)
A darkly comedic portrait of Chip Seinfeld- the degenerate, cocaine-addicted "brother" of Jerry Seinfeld. The story follows Chip through the streets of Columbus, Ohio as he struggles to break out of the shackles of his failed attempt at a stand-up comedy career. Just when it looks like he gets the chance of his life to headline a comedy festival, his paranoia gets the best of him and he shoots himself in the foot just like he has always done in the past. He might be obscure to the general public but within the world of stand-up comedy he is loved by few and hated by many.
SKU: DP-MOVIE-43228 Category: Comedy Tags: 2011, Comedy, Eddie Pepitone, Hollywood, Jake Iannarino, Jeff Gage, Mike Newman, Patton Oswalt, R, Released, stand-up comedy
A darkly comedic portrait of Chip Seinfeld- the degenerate, cocaine-addicted "brother" of Jerry Seinfeld. The story follows Chip through the streets of Columbus, Ohio as he struggles to break out of the shackles of his failed attempt at a stand-up comedy career. Just when it looks like he gets the chance of his life to headline a comedy festival, his paranoia gets the best of him and he shoots himself in the foot just like he has always done in the past. He might be obscure to the general public but within the world of stand-up comedy he is loved by few and hated by many.
Tagline: What if Jerry Seinfeld had a cokehead comedian brother?
Original Title: Who the F#ck Is Chip Seinfeld?
Certification / MPAA Film Rating: R
Runtime: 26min | 0h 26min
Jake Iannarino – Chip
Patton Oswalt –
Eddie Pepitone –
Jeff Gage – Herb
No trailers available for this title.
WOULD SHIP OUT OF PK WAREHOUSE
NON-STOCK INVENTORY, AVAILABILITY WOULD BE CHECKED UPON REQUEST
Silicon Valley Season 2 Blu-Ray 2015 (Original)
Title: Silicon Valley Season 2 Blu-Ray 2015
Label: Warner Bros
Sixteen Candles (Original)
A teenage girls deals with her parents forgetting her birthday and a crush on her high school's heartthrob.
Is That a Gun in Your Pocket? (2016)
Spurred by a gun incident at a school, the women of one Texas town lead a movement to get rid of guns by withholding sex from their husbands.
Three's definitely a crowd for parcel post deliveryman Doug Heffernan (Kevin James), whose newly widowed father-in-law, Arthur (Jerry Stiller), has moved in with him and his wife, Carrie (Leah Remini). Doug's no longer the king of his domain — the renovated basement that houses his beloved supersized TV set — let alone the king of Queens, where he lives. Can they all just get along?


Your qualifications: specifically recent professional achievements, academics, research, increases in responsibility, initiatives, and community service.
Steps you have taken to ameliorate weaknesses.
The first two areas demonstrate that you are an even better applicant today than you were when you applied. The third reveals that you belong at that school like a hand fits in a snug glove on a cold winter day, and that you will attend if, or should I say when, accepted.
Briefly thank the school for continuing to consider your application and mention how the school's philosophy and approach fit your educational preferences and goals. Don't dwell on your disappointment at not being accepted.
Agree to take any additional courses or follow any additional instructions provided.
Discuss recent achievements. Did you have a 4.0 during the last quarter? Have you led a project or organization? Volunteered? Have you taken your department, business, or club in a new direction? Have you had an article published? Earned a patent? Launched a business? Received a promotion or assumed additional responsibility? Succeeded in a particularly demanding class or project? You should bring out any recent accomplishments not discussed in your application and ideally tie them back to some of the themes or experiences you raised in your essay(s).
Discuss how you have addressed shortcomings – without highlighting them. For example, if you enrolled in Toastmasters to improve your communications skills, inform the adcom that you joined Toastmasters two months ago, tell them of any awards you have won, and enlighten them as to how much you are enjoying the experience. BUT don't say that you are doing all this because you are concerned about your low verbal score or sub-standard grades in social science courses.
If you are certain you would attend this school, make it clear that this is your first choice and that you will attend if accepted.
Keep the letter short and sweet — two pages max. Don't succumb to the temptation to rewrite or even summarize your life history or essay(s). Stay focused on what you have accomplished since applying.
The above is an excerpt from The Nine Mistakes You Don't Want to Make on an MBA Waitlist. If you are waitlisted at medical school or law school, you can find the advice you need The Nine Mistakes You Don't Want to Make on a Medical School Waitlist or The Nine Mistakes You Don't Want to Make on a Law School Waitlist.


text:
Anthony J. Guzzi
Chairman, President and Chief Executive Officer
Anthony (Tony) J. Guzzi is Chairman, President and Chief Executive Officer of EMCOR Group, Inc., (NYSE: EME), a Fortune 500? leader in mechanical and electrical construction, industrial and energy infrastructure, and building services for a diverse range of businesses and government, with 2018 revenues of $8.1 B and 36,000+ employees. For 11 consecutive years EMCOR has been named by Fortune? as one of the top five World's Most Admired Companies in the Engineering and Construction Industry.
Prior to joining EMCOR, Tony was President of the North American Distribution and Aftermarket Division of Carrier Corporation, a manufacturer and distributor of commercial and residential HVAC and refrigeration systems and equipment and a provider of aftermarket services and components of its own products and those of other manufacturers in both the HVAC and refrigeration industry. Previously, Tony served as an engagement manager with McKinsey & Company.
Tony is also Lead Director of Hubbell Incorporated.
Passionate about giving back to veterans and their families, Tony helped found the Johnny Mac Soldiers Fund. Johnny Mac's mission is to provide educational opportunities to the children and spouses of those soldiers who either perished in combat or training defending our country.
Tony is Ranger qualified and served in the U.S. Army as a Light Infantry Captain from 1986 to 1991. He graduated with a Bachelor of Science with highest distinction from The United States Military Academy at West Point in 1986. Tony earned his MBA from Harvard Business School with distinction in 1993.
Tony enjoys spending time with his family, including two sons and a daughter. He also enjoys playing golf, and attending sporting events.
Mark A. Pompa
R. Kevin Matz
Maxine Lum Mauricio
Michael J. Parry
Michael P. Bordes
Douglas R. Harrington, Jr.
Keith Chanter
Paul Desmarais
Steven H. Fried
Lisa Haight
Matthew R. Pierce
Anthony R. Triano


Immersive Technologies, the leading supplier of surface and underground operator training simulators to the global mining industry, today announced a significant milestone with the certification of its 1000th TrainerAdvantage™ participant.
This unique, globally-recognized certification program was launched by Immersive Technologies in 2006 to assist its customers to achieve a strong return on investment from their Immersive Technologies' simulators, through improved understanding and application of the technology.
Designed for simulator training supervisors, the three level certification programme includes classroom and hands-on sessions working with their simulator under the instruction of an Immersive Technologies' instructor. Each level is designed to progressively increase the trainer's knowledge, from basic operation of the simulator to advanced administration and management.
Peter Salfinger, Chief Executive Officer of Immersive Technologies' said, "We are extremely pleased by the global acceptance of this certification program. Consistent feedback from our customers confirms TrainerAdvantage™ is an important tool to help drive the value from their simulator investment that they expect."
"The training and certification of 1,000 operator trainers is a positive reflection that the industry recognizes the significant difference well-utilized Immersive Technologies' simulators can make to a site's safety and profitability."
"It also provides mines with a valuable standard to use when recruiting new training staff for their Immersive Technologies' simulators. It is common now to see mines advertising for training staff with the TrainerAdvantage™ certification."
TrainerAdvantage™ is one part of a comprehensive package of support services available to Immersive Technologies' customers. Other elements include technical support, training systems integration and analytical measurement.
Salfinger concluded, "We recognized long ago that we needed to do much more than just deliver great technology to mine sites. We work hard to listen carefully to our customers and provide intelligent and customized solutions that solve their site specific issues."
Immersive Technologies will demonstrate its latest range of surface and underground simulators this September in Sydney Australia at AIMEX 2011.


text:
Home » About DRMS » Protection and Advocacy History
The Protection and Advocacy concept was initially triggered by a series of local television news broadcasts, which Geraldo Rivera did for the ABC News affiliate in New York City. Rivera's investigative reporting exposed abuse, neglect and lack of programming at Willowbrook, a state institution for people with mental retardation on Staten Island.
These broadcasts galvanized the state's senior senator, Jacob Javitts, to action, incorporating P&A Systems in the renewal of the federal developmental disabilities legislation enacted in 1975. While their mandate was drawn more broadly in the statute, P&A Systems were originally intended to protect people in mental retardation facilities from abuse and neglect.
The National Disability Rights Network – previously the National Association of Protection and Advocacy Systems (NAPAS) – was first established in 1980 after several Executive Directors saw the need for representation in Washington on behalf of their programs.
In 1984, NAPAS worked successfully to expand the Client Assistance Program (CAP), which provides advocacy services to clients of state Vocational Rehabilitation (VR) agencies. As a result, such programs were mandated in all states, with more than half of the state P&A Systems designated to provide CAP services, as well.
That same year, Congress again renewed the developmental disabilities statute, strengthening their authority to intercede in matters of inappropriate institutional care.
Two years later, Congress designated P&A Systems to provide advocacy services to people who were disabled as a result of mental illness, as well. In the course of only ten years, federal support for advocacy on behalf of people with disabilities expanded from a narrow initial focus on the institutional care provided to people in mental retardation facilities to include advocacy services for people with mental illnesses and clients of vocational rehabilitation agencies. In a number of states, this federally funded program has attracted additional state, local, and private support to extend its services to all people with disabilities regardless of the nature or severity of those disabilities. Over the decade, NDRN has become a force for positive change in the national disability community, playing a major role in the expansion of Protection and Advocacy Services by conveying the collective concerns to the Office of Developmental Disabilities, as well as to the contractors themselves.
The yearly national P&A conferences were central to this development of the Network, as it was at these meetings that administrators first discovered the extent to which they shared common problems and learned to value the expertise they had collectively developed in meeting them. Continuing to use the subsequent reauthorizations of its programs, NDRN continued to find new opportunities to expand the eligibility and resources of the program. In a little-known section of the Rehabilitation Act, reference was made to a Protection and Advocacy System for Individual Rights. NDRN convinced RSA to demonstrate the usefulness of P&As receiving funds to provide advocacy services to all people with disabilities not eligible for the other programs ( DD, MI and CAP). After the three-year demonstration, the Rehabilitation Act was amended to create a new formula grant program (PAIR) modeled after the other programs.
In 1993, Congress reauthorized the Assistive Technology (AT) Act of 1988. The Act set up a series of state AT projects, which had come into existence over the previous five years. Based on reports from the National Council on Disability, NDRN was able to secure funding for subcontracts from the AT projects to each P&A to provide legally based advocacy services to individuals with disabilities on assistive technology issues.
In 1999, an effort was launched by the disability community to pass the Ticket to Work and Work Incentive Implementation Act (TWWIIA), which assists beneficiaries of Social Security in returning to work and includes a P&A program to assist individuals who received SSDI or SSI benefits and wish to return to work.
Two important provisions for the P&A System were included in the Children's Health Act of 2000. First the PAIMI program was expanded to cover individuals with mental illness in the community. Secondly, Congress authorized the new P&A program for people with traumatic brain injury (TBI).
Most recently, the Protection & Advocacy for Voting Access (PAVA) program was created in 2002 when Congress enacted the Help America Vote Act (HAVA). PAVA enables P&As to secure election access for a wide range of individuals with disabilities – including, but not limited to, in dividuals with mental, sensory, and physical disabilities – pursuant to Congress' broad mandate to "ensure the full participation in the electoral process for individuals with disabilities, including registering to vote, casting a vote and accessing polling places"