text:
Huichi, A Worthy Ambassador for the Huichol People? Por Hayuaneme García Mijarez
hayuaneme_garcia_mijarez.jpg
The 2011 Pan American Games, to be held in October in Guadalajara, Mexico, have three mascots. One of them is Huichi, a caricature of the sacred Huichol deer, and according to Emilio González, Jalisco state Governor, and the Games Organising Committee, a "worthy ambassador of the Huichol". However, far from being a "worthy ambassador", for the Huicholes Huichi represents a sacrilegious misuse of sacred Huichol symbology. If the government had bothered asking beforehand – which it didn't – it would have found that out. In another sleight, the Huicholes, whose artisanry is famous worldwide, formally proposed having a fixed space to sell their artisanry during the games, but this was rejected, all of this by which time Huichi had already been made public.
Languages Also Die: The Case of the Wixárika by Oscar Ukeme Bautista Muñoz
ukeme_2_-_janelle_johnson_2009.jpg
The following exhibit can be applied not only to Wixárika (Huichol) but to any indigenous language, and is particularly dedicated to those indigenous languages in danger of becoming extinct. Only due to contextual and regional reasons have we decided to describe it and compare it in this manner. Furthermore, we are not trying to say that the Wixárika language is inferior to Spanish, to the contrary, we intend to make a fitting reflection regarding the preservation of indigenous languages. Let us thus begin.
Nierika by Juan Negrín
nierika_lead_copy.jpg
The nierika is represented among the Huichol Indians of northwestern Mexico as a focal point on which powerful beings concentrate their energy. This may be as primordial as a well-crafted deer snare that induces the sacred animal's willing self-immolation. It can be a symbolic spider's web or threads attached to a wooden loop. Nierikate (plural) are holes penetrating the caves of the heart of darkness in the deep canyons and the crater of Burnt Peak, where Our Father rises from the underworld at dawn.
Sacred Sites by Juan Negrín
teacata_february_1976_web2.jpg
The Huichol tradition is rendered by three different terms: the first refers to our heart/ memory, tayeiyari- the second to how we develop, tanuiwari- the third to our life, tatukari, is transmitted by families, reinforced by communal living on extended family ranches and through clans at ceremonial centers, tukite (tukipa, sing.). Three places serve as the headquarters for what appear to be distinct Huichol subgroups, with ritual and dialectical variants. Wautüa, Tuapurie, and Tatei Kie are the ceremonial centers where traditional authorities are selected to supervise the general political and religious activities within their communities, which include the other tukite on their periphery.
Early History by Juan Negrín
sierra_madre_occidental_santa_catarina_1979_960w_150_dm.jpg
As archaeologist Phil Weigand puts it, the Huichol and their Cora neighbors had deep roots in the area where they are now settled in a sequence that had begun by the Mesoamerican Classic period (ca. 200-700 A. D.). The Corachol branch of this Uto-Aztecan language family leads linguists like Valiñas, cited by Weigand to consider the relative antiquity of this language group in the area. "This branch of Uto-Aztecan is far more closely related to the Taracahitan and Tepiman branches spoken to the north and west than to the Nahuan languages spoken farther east and south."1 Furthermore, "In prehistoric times the Huicholes, the Coras and the Tepehuanos formed a single nation with the Opatas, the Tarahumaras and the Pimas" writes historian Salvador Gutiérrez Contreras,2 confirming the opinions of earlier historians Ignacio Dávila Garibi and Alberto Santoscoy.
Post-Hispanic History by Juan Negrín
1979_tuapurie_semana_santa_960w_150.jpg
The Huichol were distinguished as xurute, according to a geographical map published in 1579, in the Atlas Theatrum Orbis Terrarum (reproduced in various sources: Rojas, Neurath). The term vizurita is used by Father Tello, in his Crónica Miscelánea, written in 1652. The first reference to the huicholes as guisoles appears in a briefing to the bishop Ruiz Colmenares (between 1640 and 1650). Father Antonio Arias y Saavedra used the terms xamucas and huitzolmes in his chronicle (1673), the first ethnological work on the Indians of this area of the Sierra Madre, according to historian Gutiérrez Contreras.
Recent History by Juan Negrín
clandestine_sawmill_-_march_1981_dm_72dpi_800w_px.jpg
The government has found it very difficult to 'civilize' the Huichol and integrate them as a dependent productive working class in the mountains. After independence was achieved from Spain, the laws of the reform passed under Benito Juárez during the 1850's, restrained the power of the Catholic Church, but they also stopped recognizing Indian colonial land rights. Soon the Huichol, Cora, Tepehuan and Mexicanero Indian groups of the Western Sierra Madre were further dispossessed of their territory by their mixed blood neighbors. They rebelled, eventually uniting under Manuel Lozada, who joined French invading forces until they were stopped at Guadalajara, Jalisco, in 1873. Many of their land holdings were disenfranchised thereafter.
juan_in_the_sierras.jpg
The Huichol: Wixárika
The Huichol (wee-CHOLE), known as huicholes in Spanish, and as Wixaritari in their own language, are recognized as one of the Mexican native cultures most resilient to outside influences. Unlike most other Indians, they did not allow Catholic priests to perform mass within the three main communities in the Huichol mountains, except sometimes on Huichol terms just before Easter, and in one community, at a couple of boarding schools.
View or Print:
Download thehuichol-wixarika.pdf (476.81 KB)
What Draws the Native Huichol to the Pacific Ocean
The pilgrimage of the Huichol natives of Northwestern Mexico, known as Wixaritari in their language, to the Ocean is a vital phenomenon. Although they mostly live a good 100 kilometers from the coast of the state of Nayarit: That is where the sparks of Our Great-Grandfather the Setting-Sun, Tatutsi Sakaimuka, disappear as he enters one of his subterranean aquatic canals, and one waits for Our Father, Tayau, or Our Creator, Taweviékame, to reappear on the other side, in the sacred desert of the east, Wirikuta.
Download what_draws_huichols_to_ocean_jn.pdf (1.43 MB)
Xikuakame Leaves His Home
Embroidered Shoulder Bag - Kutsuri
Natural brown and white wool shoulder bag woven on the traditional back-strap loom.
Brown and white shoulder bag.


text:
namespace extensions {

namespace {

namespace errors = manifest_errors;
namespace keys = manifest_keys;

std::string GetInvalidManifestKeyError(base::StringPiece key) {
  return ErrorUtils::FormatErrorMessage(errors::kInvalidManifestKey, key);
}

const char kDefaultSandboxedPageCSP[] =
    "sandbox allow-scripts allow-forms allow-popups allow-modals; "
    "script-src 'self' 'unsafe-inline' 'unsafe-eval'; child-src 'self';";
const char kDefaultExtensionPagesCSP[] =
    "script-src 'self' blob: filesystem:; "
    "object-src 'self' blob: filesystem:;";
const char kDefaultSecureCSP[] = "script-src 'self'; object-src 'self';";

}  // namespace

using CSPInfoUnitTest = ManifestTest;

TEST_F(CSPInfoUnitTest, SandboxedPages) {
  // Sandboxed pages specified, no custom CSP value.
  scoped_refptr<Extension> extension1(
      LoadAndExpectSuccess("sandboxed_pages_valid_1.json"));

  // No sandboxed pages.
  scoped_refptr<Extension> extension2(
      LoadAndExpectSuccess("sandboxed_pages_valid_2.json"));

  // Sandboxed pages specified with a custom CSP value.
  scoped_refptr<Extension> extension3(
      LoadAndExpectSuccess("sandboxed_pages_valid_3.json"));

  // Sandboxed pages specified with wildcard, no custom CSP value.
  scoped_refptr<Extension> extension4(
      LoadAndExpectSuccess("sandboxed_pages_valid_4.json"));

  // Sandboxed pages specified with filename wildcard, no custom CSP value.
  scoped_refptr<Extension> extension5(
      LoadAndExpectSuccess("sandboxed_pages_valid_5.json"));

  // Sandboxed pages specified for a platform app with a custom CSP.
  scoped_refptr<Extension> extension6(
      LoadAndExpectSuccess("sandboxed_pages_valid_6.json"));

  // Sandboxed pages specified for a platform app with no custom CSP.
  scoped_refptr<Extension> extension7(
      LoadAndExpectSuccess("sandboxed_pages_valid_7.json"));

  const char kCustomSandboxedCSP[] =
      "sandbox; script-src 'self'; child-src 'self';";

  EXPECT_EQ(kDefaultSandboxedPageCSP, CSPInfo::GetResourceContentSecurityPolicy(
                                          extension1.get(), "/test"));
  EXPECT_EQ(
      kDefaultExtensionPagesCSP,
      CSPInfo::GetResourceContentSecurityPolicy(extension1.get(), "/none"));
  EXPECT_EQ(
      kDefaultExtensionPagesCSP,
      CSPInfo::GetResourceContentSecurityPolicy(extension2.get(), "/test"));
  EXPECT_EQ(kCustomSandboxedCSP, CSPInfo::GetResourceContentSecurityPolicy(
                                     extension3.get(), "/test"));
  EXPECT_EQ(
      kDefaultExtensionPagesCSP,
      CSPInfo::GetResourceContentSecurityPolicy(extension3.get(), "/none"));
  EXPECT_EQ(kDefaultSandboxedPageCSP, CSPInfo::GetResourceContentSecurityPolicy(
                                          extension4.get(), "/test"));
  EXPECT_EQ(kDefaultSandboxedPageCSP, CSPInfo::GetResourceContentSecurityPolicy(
                                          extension5.get(), "/path/test.ext"));
  EXPECT_EQ(
      kDefaultExtensionPagesCSP,
      CSPInfo::GetResourceContentSecurityPolicy(extension5.get(), "/test"));
  EXPECT_EQ(kCustomSandboxedCSP, CSPInfo::GetResourceContentSecurityPolicy(
                                     extension6.get(), "/test"));
  EXPECT_EQ(kDefaultSandboxedPageCSP, CSPInfo::GetResourceContentSecurityPolicy(
                                          extension7.get(), "/test"));

  Testcase testcases[] = {
      Testcase("sandboxed_pages_invalid_1.json",
               errors::kInvalidSandboxedPagesList),
      Testcase("sandboxed_pages_invalid_2.json", errors::kInvalidSandboxedPage),
      Testcase("sandboxed_pages_invalid_3.json",
               GetInvalidManifestKeyError(keys::kSandboxedPagesCSP)),
      Testcase("sandboxed_pages_invalid_4.json",
               GetInvalidManifestKeyError(keys::kSandboxedPagesCSP)),
      Testcase("sandboxed_pages_invalid_5.json",
               GetInvalidManifestKeyError(keys::kSandboxedPagesCSP))};
  RunTestcases(testcases, base::size(testcases), EXPECT_TYPE_ERROR);
}

TEST_F(CSPInfoUnitTest, CSPStringKey) {
  scoped_refptr<Extension> extension =
      LoadAndExpectSuccess("csp_string_valid.json");
  ASSERT_TRUE(extension);
  EXPECT_EQ("script-src 'self'; default-src 'none';",
            CSPInfo::GetExtensionPagesCSP(extension.get()));

  // Manifest V2 extensions bypass the main world CSP in their isolated worlds.
  const std::string* isolated_world_csp =
      CSPInfo::GetIsolatedWorldCSP(*extension);
  ASSERT_TRUE(isolated_world_csp);
  EXPECT_TRUE(isolated_world_csp->empty());

  RunTestcase(Testcase("csp_invalid_1.json", GetInvalidManifestKeyError(
                                                 keys::kContentSecurityPolicy)),
              EXPECT_TYPE_ERROR);
}

TEST_F(CSPInfoUnitTest, CSPDictionary_ExtensionPages) {
  struct {
    const char* file_name;
    const char* csp;
  } cases[] = {{"csp_dictionary_valid_1.json", "default-src 'none'"},
               {"csp_dictionary_valid_2.json",
                "worker-src 'self'; script-src; default-src 'self'"},
               {"csp_empty_dictionary_valid.json", kDefaultSecureCSP}};

  for (const auto& test_case : cases) {
    SCOPED_TRACE(base::StringPrintf("Testing %s.", test_case.file_name));
    scoped_refptr<Extension> extension =
        LoadAndExpectSuccess(test_case.file_name);
    ASSERT_TRUE(extension.get());
    EXPECT_EQ(test_case.csp, CSPInfo::GetExtensionPagesCSP(extension.get()));
  }

  Testcase testcases[] = {
      Testcase("csp_invalid_2.json",
               GetInvalidManifestKeyError(
                   keys::kContentSecurityPolicy_ExtensionPagesPath)),
      Testcase("csp_invalid_3.json",
               GetInvalidManifestKeyError(
                   keys::kContentSecurityPolicy_ExtensionPagesPath)),
      Testcase(
          "csp_missing_src.json",
          ErrorUtils::FormatErrorMessage(
              errors::kInvalidCSPMissingSecureSrc,
              keys::kContentSecurityPolicy_ExtensionPagesPath, "script-src")),
      Testcase("csp_insecure_src.json",
               ErrorUtils::FormatErrorMessage(
                   errors::kInvalidCSPInsecureValueError,
                   keys::kContentSecurityPolicy_ExtensionPagesPath,
                   "'unsafe-eval'", "worker-src")),
      Testcase("csp_dictionary_with_wasm.json",
               ErrorUtils::FormatErrorMessage(
                   errors::kInvalidCSPInsecureValueError,
                   keys::kContentSecurityPolicy_ExtensionPagesPath,
                   "'wasm-eval'", "worker-src")),
  };
  RunTestcases(testcases, base::size(testcases), EXPECT_TYPE_ERROR);
}

TEST_F(CSPInfoUnitTest, AllowWasmInMV3) {
  base::test::ScopedFeatureList feature_list(
      extensions_features::kAllowWasmInMV3);

  const char kDefaultSecureCSPWithWasmAllowed[] =
      "script-src 'self' 'wasm-eval'; object-src 'self';";

  struct {
    const char* file_name;
    const char* csp;
  } cases[] = {
      {"csp_dictionary_with_wasm.json",
       "worker-src 'self' 'wasm-eval'; default-src 'self'"},
      {"csp_dictionary_empty_v3.json", kDefaultSecureCSPWithWasmAllowed},
      {"csp_dictionary_valid_1.json", "default-src 'none'"},
      {"csp_omitted_mv2.json", kDefaultExtensionPagesCSP}};

  for (const auto& test_case : cases) {
    SCOPED_TRACE(base::StringPrintf("Testing %s.", test_case.file_name));
    scoped_refptr<Extension> extension =
        LoadAndExpectSuccess(test_case.file_name);
    ASSERT_TRUE(extension.get());
    EXPECT_EQ(test_case.csp, CSPInfo::GetExtensionPagesCSP(extension.get()));
  }
}

TEST_F(CSPInfoUnitTest, CSPDictionary_Sandbox) {
  ScopedCurrentChannel channel(version_info::Channel::UNKNOWN);

  const char kCustomSandboxedCSP[] =
      "sandbox; script-src 'self'; child-src 'self';";
  const char kCustomExtensionPagesCSP[] = "script-src; object-src;";

  struct {
    const char* file_name;
    const char* resource_path;
    const char* expected_csp;
  } success_cases[] = {
      {"sandbox_dictionary_1.json", "/test", kCustomSandboxedCSP},
      {"sandbox_dictionary_1.json", "/index", kDefaultSecureCSP},
      {"sandbox_dictionary_2.json", "/test", kDefaultSandboxedPageCSP},
      {"sandbox_dictionary_2.json", "/index", kCustomExtensionPagesCSP},
  };

  for (const auto& test_case : success_cases) {
    SCOPED_TRACE(base::StringPrintf("%s with path %s", test_case.file_name,
                                    test_case.resource_path));
    scoped_refptr<Extension> extension =
        LoadAndExpectSuccess(test_case.file_name);
    ASSERT_TRUE(extension);
    EXPECT_EQ(test_case.expected_csp,
              CSPInfo::GetResourceContentSecurityPolicy(
                  extension.get(), test_case.resource_path));
  }

  Testcase testcases[] = {
      {"sandbox_both_keys.json", errors::kSandboxPagesCSPKeyNotAllowed},
      {"sandbox_csp_with_dictionary.json",
       errors::kSandboxPagesCSPKeyNotAllowed},
      {"sandbox_invalid_type.json",
       GetInvalidManifestKeyError(
           keys::kContentSecurityPolicy_SandboxedPagesPath)},
      {"unsandboxed_csp.json",
       GetInvalidManifestKeyError(
           keys::kContentSecurityPolicy_SandboxedPagesPath)}};
  RunTestcases(testcases, base::size(testcases), EXPECT_TYPE_ERROR);
}

// Ensures that using a dictionary for the keys::kContentSecurityPolicy manifest
// key is mandatory for manifest v3 extensions and that defaults are applied
// correctly.
TEST_F(CSPInfoUnitTest, CSPDictionaryMandatoryForV3) {
  LoadAndExpectError("csp_invalid_type_v3.json",
                     GetInvalidManifestKeyError(keys::kContentSecurityPolicy));

  const char* default_case_filenames[] = {"csp_dictionary_empty_v3.json",
                                          "csp_dictionary_missing_v3.json"};

  for (const char* filename : default_case_filenames) {
    SCOPED_TRACE(filename);
    scoped_refptr<Extension> extension = LoadAndExpectSuccess(filename);
    ASSERT_TRUE(extension);

    const std::string* isolated_world_csp =
        CSPInfo::GetIsolatedWorldCSP(*extension);
    ASSERT_TRUE(isolated_world_csp);
    EXPECT_EQ(kDefaultSecureCSP, *isolated_world_csp);

    EXPECT_EQ(kDefaultSandboxedPageCSP,
              CSPInfo::GetSandboxContentSecurityPolicy(extension.get()));
    EXPECT_EQ(kDefaultSecureCSP,
              CSPInfo::GetExtensionPagesCSP(extension.get()));
  }
}

// Ensure the CSP dictionary is disallowed for mv2 extensions.
TEST_F(CSPInfoUnitTest, CSPDictionaryDisallowedForV2) {
  LoadAndExpectError("csp_dictionary_mv2.json",
                     GetInvalidManifestKeyError(keys::kContentSecurityPolicy));
}

}  // namespace extensions

redpajama_set_name: RedPajamaGithub


text:
\section{Introduction} \label{sec:introduction}
Inference of parameters in a probabilistic model is an essential ingredient in model-based statistical approaches, both in the frequentist and Bayesian paradigms.
Given a probabilistic model $P(y|\theta)$, which is a conditional distribution of observations $y$ given a parameter $\theta$, the aim is to make inference about the parameter $\theta^*$ that generated an observed data $y^*$.
When the model $P(y|\theta)$ admits a conditional density $\ell(y|\theta)$, such an inference can be made on the basis of evaluations of $\ell(y^*|\theta)$; this is the {\em likelihood} of $y^*$ as a function of $\theta$.
However, in modern scientific and engineering problems in which the model $P(y|\theta)$ is required to be sophisticated and complex, the likelihood function $\ell(y^*|\theta)$ might no longer be available.
This may be because the density form of $P(y|\theta)$ is elusive, or the evaluation of the likelihood $\ell(y^*|\theta)$ is computationally very expensive.
Such situations, in which $\ell(y|\theta)$ (or $P(y|\theta)$) are referred to as {\em intractable likelihood}, make the inference problem quite challenging and are commonly found in the literature on population genetics \citep{pritchard1999population} and dynamical systems \citep{toni2009approximate}, to name just two.

{\em Approximate Bayesian Computation} (ABC) is a class of computational methods for Bayesian inference with intractable likelihood \citep{tavare1997inferring,pritchard1999population,beaumont2002approximate} that is applicable as long as {\em sampling} from the model $P(y|\theta)$ is possible.
Given a prior $\pi(\theta)$ on the parameter space, the basic ABC constructs a Monte Carlo approximation to the posterior $P_{y^*}(\theta) \propto P(y^*|\theta) \pi(\theta)$ in the following way:
i) sample pairs $(y_i,\theta_i)$ of pseudo data $y_i$ and parameter $\theta_i$ from the joint distribution $P(y|\theta)\pi(\theta)$, where $i = 1,\dots,n$ for some $n \in \mathbb{N}$, ii) maintain only those parameters $\theta_i$ associated with $y_i$ that are ``close enough'' to the observed data $y^*$, and iii) regard them as samples from the posterior $P_{y^*}(\theta)$.
ABC has been extensively studied in statistics and machine learning; see, e.g.,~ \citet{del2012adaptive,fukumizu2013kernel,meeds2014gps,pmlr-v51-park16,mitrovic2016dr}.

In this paper, we rather take the frequentist perspective, and deal with the problem of {\em maximum likelihood estimation (MLE)} with intractable likelihood.
That is, we consider situations in which one believes that there is a ``true'' parameter $\theta^*$ that generated the data $y^*$ and wishes to obtain a point estimate for it.
This problem is also motivated by the following situations encountered in practice:
1) Consider a situation in which the model is computationally expensive (e.g., a state-space model) and one wants to perform prediction based on it.
In this case fully Bayesian prediction would require simulation from each of sampled parameters, which might be quite costly.
If one has a point estimate of the true parameter $\theta^*$, then the computational cost can be drastically reduced.
2) Consider a situation in which one only has limited knowledge w.r.t. model parameters.
In this case, it is generally difficult to specify an appropriate prior distribution over the parameter space, and thus the resulting posterior may not be reliable.\footnote{For point estimation, one may think of using the maximum a posterior (MAP) estimate, but it may again be unreliable (as for the posterior distribution itself), if the prior distribution cannot be specified appropriately.}
Methods for point estimation with intractable likelihood have been reported in the literature, including the method of simulated-moments \citep{mcfadden1989method}, indirect inference \citep{gourieroux1993indirect}, ABC-based MAP estimation \citep{rubio2013simple}, noisy ABC-MLE \citep{dean2014parameter,yildirim2015parameter}, an approach based on Bayesian optimization \citep{gutmann2016bayesian}, and data-cloning ABC \citep{picchini2017approximate}.
We will discuss these existing approaches in Sec.~\ref{sec:experiments}.

Our contribution is in proposing a novel approach to point estimation with intractable likelihood on the basis of {\em kernel mean embedding of distributions}  \citep{MuaFukSriSch17}, a framework for statistical inference using reproducing kernel Hilbert spaces.
Specifically, our approach extends {\em kernel ABC} \citep{fukumizu2013kernel,nakagome2013kernel}, a method for ABC using kernel embedding of conditional distributions \citep{song2009hilbert,SonFukGre13}, to point estimation with intractable likelihood.
The novelty lies in combining kernel ABC with {\em kernel herding} \citep{chen2010super}, a deterministic sampling method similar to quasi-Monte Carlo \citep{DicKuoSlo13}, and in applying these two methods iteratively to the same observed data in a recursive way.
We term this approach {\em kernel recursive ABC}.
A theoretical explanation will be provided for this approach, discussing how such recursion yields a point estimate for the true parameter.
We also discuss that the combination of kernel ABC and kernel herding leads to robustness against misspecification of a prior for the true parameter; this is an advantage over existing methods, and will be demonstrated experimentally.

This paper is organized as follows.
We briefly review kernel ABC and kernel herding in Sec.~\ref{sec:background} and propose kernel recursive ABC in Sec.~\ref{sec:proposed}.
We report experimental results of comparisons with existing methods in Sec.~\ref{sec:experiments}.
The experiments include parameter estimation for a real-world pedestrian flow simulator \citep{Noda2010p}, which may be of independent interest as application.

\section{Background} \label{sec:background}

\subsection{Kernel ABC} \label{sec:kernelABC}
Kernel ABC is an algorithm that executes ABC in a reproducing kernel Hilbert space (RKHS) and produces a reliable solution even in moderately large dimensional problems \citep{fukumizu2013kernel,nakagome2013kernel}.
It is based on the framework of {\em kernel mean embeddings}, in which all probability measures are represented as elements in an RKHS (see \citet{MuaFukSriSch17} for a recent survey of this field).
Let $\Theta$ be a measurable space, $k: \Theta \times \Theta \to \mathbb{R}$ be a measurable positive definite kernel, and $\mathcal{H}$ be its RKHS.
In this framework, any probability measure $P$ on $\Theta$ will be represented as a Bochner integral
\begin{equation} \label{eq:kernel_mean}
\mu_P := \int_\Theta k(\cdot, \theta)dP(\theta) \in \mathcal{H},
\end{equation}
which is called the {\em kernel mean} of $P$.
If the mapping $P \to \mu_P$ is injective, in which case $\mu_P$ preserves all the information in $P$, the kernel $k$ is referred to as being {\em characteristic} \citep{fukumizu2008kernel}.
Characteristic kernels on $\Theta = \mathbb{R}^d$, for example, include Gaussian and Mat\'ern kernels \citep{SriGreFukSchetal10}.

Let $\mathcal{Y}$ be another measurable space and assume that an observed data $y^*\in \mathcal{Y}$ is provided. ($y^*$ is often a set of sample points.)
Given a conditional probability $P(y|\theta)$ and a prior $\pi(\theta)$, we wish to obtain the posterior distribution $P_{y^*}(\theta) \propto P(y^*|\theta)\pi(\theta)$.\footnote{There is abuse of notation here, as $P(y|\theta)$ does not denote a conditional density but a conditional distribution.}
As in a standard ABC, kernel ABC achieves this by first generating pairs of pseudo data and parameter $\{ (y_i,  \theta_{i}) \}_{i=1}^{n}$ from the joint distribution $P(y |\theta)\pi(\theta)$.
It then estimates the kernel mean of the posterior $P_{y^*}$, which we denote by
$$\mu_{P_{y^*}} := \int_\Theta k (\cdot,\theta)dP_{y^*}(\theta) \in \mathcal{H}.$$
Given a measurable positive definite kernel $k_{\mathcal{Y}}$ on $\mathcal{Y}$, the estimator is given by
\begin{eqnarray}
\hat{\mu}_{P_{y^*}} &=& \sum^{n}_{i=1}w_i k(\cdot , \theta_i) \in \mathcal{H}, \label{eq:cond_kmean} \\
{\bm w} &:=& (w_1,\dots,w_n)^T := (G + n \delta I)^{-1} {\bm k}(y^*), \label{eq:KRR_weight}
\end{eqnarray}
where ${\bm k}(y^*) := (k_\mathcal{Y}(y_1,y^*), \dots, k_\mathcal{Y}(y_n, y^*))^T \in \mathbb{R}^n$, $G := (k_\mathcal{Y}(y_i,y_j))_{i,j=1}^n \in \mathbb{R}^{n \times n}$, $\delta > 0$ is a regularization constant, and $I \in \mathbb{R}^{n \times n}$ is an identity matrix.
The estimator (\ref{eq:cond_kmean}) is essentially an (RKHS-valued) kernel ridge regression \citep{GruLevBalPatetal12}: Given {\em training} data $\{ (y_i,  k(\cdot,\theta_{i}) ) \}_{i=1}^{n}$, the weights (\ref{eq:KRR_weight}) provide an estimator for the mapping $y^* \Rightarrow k(\cdot,\theta^*)$.
For consistency and convergence results, which require $\delta \to 0$ as $n \to \infty$, we re refer to \citet{fukumizu2013kernel} and \citet{MuaFukSriSch17}.

\subsection{Kernel herding} \label{sec:kernelherding}

Kernel herding is a deterministic sampling technique based on the kernel mean representation of a distribution \citep{chen2010super} and can be seen as a greedy approach to quasi-Monte Carlo \citep{DicKuoSlo13}.
Consider sampling from $P$ using the kernel mean $\mu_P$ (\ref{eq:kernel_mean}), and assume that one is able to evaluate function values of $\mu_P$.
Kernel herding greedily obtains sample points $\theta_1,\theta_2,\dots, \theta_n$ by iterating the following steps: Defining $h_{0} := \mu_P$,
\begin{eqnarray}
\theta_{t+1} &=& \operatornamewithlimits{argmax}_{\theta \in \Theta}  h_t(\theta), \label{eq:herding1}\\
h_{t+1} &=& h_t + \mu_P - k(\cdot , \theta_{t+1}) \in \mathcal{H}, \label{eq:herding2}
\end{eqnarray}
where $t = 0,\dots,n-1$.
\citet{chen2010super} has shown that, if there exists a constant $C > 0$ such that $k(\theta, \theta) = C$ for all $\theta \in \Theta$, this procedure will be identical to the greedy minimization of the maximum mean discrepancy (MMD) \citep{gretton2007kernel,gretton2012kernel}:
\begin{equation} \label{eq:MMD}
\epsilon_n := \left\| \mu_P - \frac{1}{n}\sum^n_{t=1} k(\cdot, \theta_t) \right\|_\mathcal{H},
\end{equation}
where $\| \cdot \|_\mathcal{H}$ denotes the norm of $\H$.
That is, the points $\theta_1,\dots,\theta_n$ are obtained so as to (greedily) minimize the distance $\varepsilon_n$ between $\mu_P$ and the empirical kernel mean $\frac{1}{n}\sum^n_{t=1}k(\cdot , \theta_t)$.
The generated points $\theta_1,\dots,\theta_n$ are also called super-samples because they are more informative than those from random sampling; this is in the sense that error decreases at the rate $\epsilon_{n} = O(n^{-1})$ if the RKHS is finite-dimensional \citep{bach2012equivalence}, which is faster than the rate $\epsilon_{n} = O(n^{-1/2})$ of random sampling \citep{SmoGreSonSch07}.
Convergence guarantees are also provided even when the optimization problem in (\ref{eq:herding1}) is solved approximately \citep{LacLinBac15} and when the kernel mean $\mu_P$ is replaced by an empirical estimate $\hat{\mu}_P$ of the form (\ref{eq:cond_kmean})  \citep{kanagawa2016filtering}.
Note that the decay $\epsilon_n \to 0$ of the error (\ref{eq:MMD}) as $n \to \infty$ implies the convergence of expectation $\frac{1}{n} \sum_{t=1}^n f(\theta_t) \to \int f(x)dP(x)$ for all functions $f$ in the RKHS $\H$ and for functions $f$ that can be approximated well by the RKHS functions \citep{KanSriFuk16}.

\section{Proposed method} \label{sec:proposed}
\label{sec:proposed-algorithm}

\begin{algorithm}[tb]
   \caption{Kernel Recursive ABC}
   \label{kr-abc}
\begin{algorithmic}
   \STATE {\bfseries Input:} A prior distribution $\pi$, an observed data $y^*$, a data generator $P(y|\theta)$, the number  $N_{\rm iter}$ of iterations, the number $n$ of simulated pairs, a kernel $k$ on $\Theta$, a kernel $k_\mathcal{Y}$ on $\mathcal{Y}$, and a regularization constant $\delta > 0$
   \STATE {\bfseries Output:} A point estimate $\acute{\theta}$.
   \FOR{\texttt{$N = 1,..., N_{\rm iter}$}}
   \IF{$N=1$}{
    	\FOR{\texttt{$i = 1,..., n$}}
        \STATE
    	Sample $\theta_{1,i} \sim \pi (\theta)$ i.i.d.
        \ENDFOR
        }
    \ENDIF
    \FOR{\texttt{$i = 1,..., n$}}
    \STATE Generate $y_{N,i} \sim P(\cdot | 		\theta_{N,i})$
    \ENDFOR
    \STATE Compute $G := (k_\mathcal{Y}(y_{N,i}, y_{N,i}))_{i,j=1}^n \in \mathbb{R}^{n \times n}$ and ${\bm k}(y) := (k_\mathcal{Y}(y_{N,i}, y^*))_{i=1}^n \in \mathbb{R}^n$.
    \STATE Calculate ${\bm w} = (w_1,\dots,w_n)^T \in \mathbb{R}^n$ by Eq.(\ref{eq:KRR_weight}).
    \STATE Construct a kernel mean estimate of the powered posterior $\hat{\mu}_{P_N} := \sum_{i=1}^{n} w_i k(\cdot, \theta_{N,i})$
    \STATE Sample $\{\theta_{N+1,t}\}_{t=1}^n$ by performing kernel herding Eqs.(\ref{eq:herding1}) (\ref{eq:herding2}) with $\mu_P := \hat{\mu}_{P_N}$.
\ENDFOR
\STATE Obtain a point estimate $\acute{\theta} := \theta_{N_{\rm iter}+1,1}$
\end{algorithmic}
\end{algorithm}

\begin{comment}
\begin{algorithm}[t]
\caption{Kernel Recursive ABC}
	\label{kr-abc}
    \SetKwInOut{Input}{Input}
    \SetKwInOut{Output}{Output}
    \Input{A prior distribution $\pi$, an observed data $y^*$, a data generator $P(y|\theta)$, the number  $N_{\rm iter}$ of iteration, the number $n$ of input-output pairs, a kernel $k$ on $\Theta$, a kernel $k_\mathcal{Y}$ on $\mathcal{Y}$ and a regularization constant $\delta > 0$.}
    \Output{A maximum likelihood estimate $\acute{\theta}$.}
    \For{\texttt{$N = 1,..., N_{\rm iter}$}}{
    \If{N=1}{
    	\For{\texttt{$i = 1,..., n$}}
        {
    	Sample $\theta_{1,i} \sim \pi (\theta)$ i.i.d.
      }
        }
    \For{\texttt{$i = 1,..., n$}}{
        Generate $y_{N,i} \sim P(\cdot | \theta_{N,i})$}
    Compute $G := (k_\mathcal{Y}(y_{N,i}, y_{N,i}))_{i,j=1}^n \in \mathbb{R}^{n \times n}$ and ${\bm k}(y) := (k_\mathcal{Y}(y_{N,i}, y^*))_{i=1}^n \in \mathbb{R}^n$, and calculate ${\bm w} = (w_1,\dots,w_n)^T \in \mathbb{R}^n$ by Eq.(\ref{eq:KRR_weight}). \\
    Construct a kernel mean estimate of the powered posterior $\hat{\mu}_{P_N} := \sum_{i=1}^{n} w_i k(\cdot, \theta_{N,i})$\\
    Sample $\{\theta_{N+1,t}\}_{t=1}^n$ by performing kernel herding Eqs.(\ref{eq:herding1}) (\ref{eq:herding2}) with $\mu_P := \hat{\mu}_{P_N}$. }
Obtain a point estimate $\acute{\theta} := \theta_{N_{\rm iter}+1,1}$
\end{algorithm}
\end{comment}
Our idea is to recursively apply Bayes' rule to the same observed data $y^*$ by using the posterior obtained in one iteration as a prior for the next iteration.
For this, let $\ell(\theta) := \ell(y^*|\theta)$ be a likelihood function and $\pi(\theta)$ be a prior density, where $\theta \in \Theta$, with $\Theta$ being a measurable space.
Consider the population setting in which no estimation procedure is involved.
After the $N$-th recursion, the posterior distribution becomes
\begin{equation}  \label{eq:power_post}
p_N (\theta) := C_N^{-1}\ \pi(\theta) (\ell(\theta))^N,
\end{equation}
where $C_N := \int_\Theta \pi(\theta) \left( \ell(\theta) \right)^N d\theta$ is a normalization constant.
We refer here to this as a {\em powered posterior}.
If $\ell$ has a unique global maximum at $\theta_{\infty} \in \Theta$ and the support of $\pi$ contains $\theta_\infty$, one can show that $p_N$ converges weakly to the Dirac distribution $\delta_{\theta_{\infty}}$ at $\theta_\infty$ under certain conditions \citep{LelNadSch10}.
In other words, the effect of the prior diminishes as the recursion proceeds, and the powered posterior degenerates at the maximum likelihood point, providing a method for MLE.
A similar idea has been discussed by \citet{doucet2002marginal,LelNadSch10} in the context of data augmentation and data cloning, in which one replicates the observed data $y^*$ multiple times and applies Bayes' rule once; our approach is different, as we employ recursive applications of Bayes' rule multiple times (this turns out to be beneficial in our approach, as is shown below).

Based on the above idea, we propose to recursively applying kernel ABC (Sec.~\ref{sec:kernelABC}) and kernel herding (Sec.~\ref{sec:kernelherding}).
Specifically, the proposed method (Algorithm \ref{kr-abc}) iterates the following procedures:
(i) At the $N$-th iteration, the kernel mean $\mu_{P_N} := \int k(\cdot,\theta)p_N(\theta)d\theta$ of the powered posterior (\ref{eq:power_post}) is estimated using simulated pairs $\{(\theta_{N,i},y_{N,i})\}_{i=1}^n$ via kernel ABC;
(ii) from the estimate $\hat{\mu}_{P_N}$ of $\mu_{P_N}$ given in (i), new parameters $\{ \theta_{N+1,i} \}_{i=1}^n$ are generated via kernel herding, and new pseudo-data $\{ y_{N+1,i} \}_{i=1}^n$ are generated from the simulator $P(y_{N+1,i}|\theta_{N+1,i})$ in the $N+1$-th iteration.
After iterating these procedures $N_{\rm iter}$ times, point estimate $\acute{\theta}$ for the true parameter is given as the first point $\theta_{N_{\rm iter}+1,1}$ from kernel herding at the last iteration.

\vspace{-0.3cm}
\paragraph{Auto-correction mechanism.}
An interesting feature of the proposed approach is that, as experimentally indicated in Sec.~\ref{sec:Gauss-misspecified}, it is equipped with an auto-correction mechanism: If the parameters $\theta_{N,1},\dots,\theta_{N,n}$ at the $N$-th iteration are far apart from the true parameter $\theta^*$, then Algorithm \ref{kr-abc} searches for the parameters $\theta_{N+1,1},\dots,\theta_{N+1,n}$ at the next iteration, so as to explore the parameter space $\Theta$.
For instance, if the prior $\pi(\theta)$ is misspecified, meaning that the true parameter $\theta^*$ is not contained in the support of $\pi(\theta)$, then the initial parameters $\theta_{1,1},\dots,\theta_{1,n}$ from  $\pi(\theta)$ are likely to be apart from the true parameter $\theta^*$.
The auto-correction mechanism makes the proposed method robust to such misspecification and makes it suitable for use in situations in which one lacks appropriate prior knowledge about the true parameter.

To explain how this works, let us explicitly write down the procedure (\ref{eq:herding1}) (\ref{eq:herding2}) of kernel herding as used in Algorithm \ref{kr-abc}.
Given that $t\ (< n)$ points $\theta_{N+1,1},\dots,\theta_{N+1,t}$ have already been generated, the next point $\theta_{N+1,t+1}$ is obtained as
\begin{eqnarray} \label{eq:herding_alg}
&&\theta_{N+1,t+1} := \\
&& \operatornamewithlimits{argmax}_{\theta \in \Theta} \sum_{i=1}^{n} w_i k(\theta, \theta_{N,i}) -  \dfrac {1}{t+1}\sum^t_{i=1}k(\theta , \theta_{N+1,i}), \nonumber
\end{eqnarray}
where the weights $w_1,\dots,w_n$ are given as (\ref{eq:KRR_weight}).
Assume that all the simulated parameters $\theta_{N,1},\dots,\theta_{N,n}$ at the $N$-th iteration are far apart from the true parameter $\theta^*$: If $N=1$, these are the parameters sampled from the prior $\pi(\theta)$.
Then it is likely the resulting simulated data $y_{N,1},\dots,y_{N,n}$ are dissimilar to the observed data $y^*$.
In this case, each component of the vector ${\bm k}(y) := (k_\mathcal{Y}(y_{N,i}, y^*))_{i=1}^n \in \mathbb{R}^n$  becomes nearly $0$, since $k_\mathcal{Y}(y_{N,i}, y^*)$ quantifies the similarity between $y^*$ and $y_{N,i}$.
As a result, each of the weights $w_1,\dots,w_n$ given by kernel ABC (\ref{eq:KRR_weight}) also become nearly $0$, and thus the first term on the right side in (\ref{eq:herding_alg}) will be ignorable.
The point $\theta_{N+1,{t+1}}$ is then obtained so as to roughly maximize the second term $-\frac{1}{t+1} \sum_{i=1}^t k(\theta_{N+1,{t+1}},\theta_{N+1,i})$, or, equivalently, so as to minimize $ \sum_{i=1}^t k(\theta_{N+1,{t+1}},\theta_{N+1,i})$.
Since the kernel $k(\theta_{N+1,{t+1}}, \theta_{N+1,i})$ measures the similarity between $\theta_{N+1,{t+1}}$ and $\theta_{N+1,i}$, the new point $\theta_{N+1,{t+1}}$ is located apart from the points $\theta_{N+1,1},\dots,\theta_{N+1,t}$ generated so far.
In this way, the parameters $\theta_{N+1,1},\dots,\theta_{N+1,n}$ at the $N+1$-th iteration are made to explore the parameter space $\Theta$ if parameters $\theta_{N,1},\dots,\theta_{N,n}$ at the $N$-th iteration are far apart from the true parameter $\theta^*$.

\subsection{Theoretical analysis} \label{sec:theory}
We provide here a theoretical basis for the proposed recursive approach.
Since the consistency of kernel ABC and kernel herding have already been established in the literature \citep{fukumizu2013kernel,bach2012equivalence}, we focus  on convergence analysis in the population setting, that is, convergence analysis for the kernel mean of the powered posterior (\ref{eq:power_post}) and for the resulting point estimate.
We nevertheless note that convergence analysis of the overall procedure of Algorithm \ref{kr-abc} remains an important topic for future research.
All the proofs can be found in the Supplementary Materials.

Below, we let $\Theta$ be a Borel measurable set in $\mathbb{R}^d$.
Denote by $P_N$ the probability measure induced by the powered posterior density $p_N$ (\ref{eq:power_post}), and let $\mu_{P_N} := \int k(\cdot,\theta)dP_N(\theta) \in \mathcal{H}$ be its kernel mean, where $k$ is a kernel on $\Theta$ and $\mathcal{H}$ is its RKHS.
We require the following assumption for the likelihood function $\ell$ and the prior $\pi$ for theoretical analysis.
\begin{assumption} \label{as:likelihood_prior}
(i) $\ell$ has a unique global maximum at $\theta_\infty \in \Theta$, and $\pi(\theta_\infty) > 0$;
(ii) $\pi$ is continuous at $\theta_\infty$, $\ell$ has continuous second derivatives in the neighborhood of $\theta_\infty$, and the Hessian of $\ell$ at $\theta_\infty$ is strictly negative-definite.
\end{assumption}

Our first result below shows that, under Assumption \ref{as:likelihood_prior}, the powered posterior $P_N$ (\ref{eq:power_post}) converges to the Dirac distribution $\delta_{\theta_\infty}$ in the RKHS $\mathcal{H}$ as $N \to \infty$;
this provides a theoretical basis for recursively applying the kernel ABC.
\begin{proposition} \label{prop:convergence}
Let $\Theta \subset \mathbb{R}^d$ be a Borel measurable set and $k: \Theta \times \Theta \to \mathbb{R}$ be a continuous bounded kernel.
Under Assumption \ref{as:likelihood_prior}, we have
$\lim_{N \to \infty} \left\| \mu_{P_N} - k(\cdot,\theta_\infty) \right\|_{\mathcal{H}} = 0$.
\end{proposition}
Proposition \ref{pr:consistency2} below provides a justification for the use of the first point of kernel herding (here this is $\theta_N := \operatornamewithlimits{argmin}_{\tilde{\theta} \in \Theta} \| \mu_{P_N} - k(\cdot,\tilde{\theta}) \|_{\mathcal{H}}$; see Sec.~\ref{sec:kernelherding}) as a point estimate of $\theta_\infty$.
To this end, we introduce the following assumption on the kernel, which is satisfied by, for example, Gaussian and Mat\'ern kernels.
\begin{assumption} \label{as:kernel}
(i) There exists a constant $C > 0$ such that $k(\theta,\theta) = C$ for all $\theta \in \Theta$.
(ii)  It holds that $k(\theta,\theta') < C$ for all $\theta, \theta' \in \Theta$ with $\theta \neq \theta'$.
\end{assumption}

\begin{comment}
\begin{proof}
Because Assumption \ref{as:likelihood_prior} is equivalent to Assumptions A1, A2, and A3 in \citep{LelNadSch10}, we can use the Corollary to Lemma A.2 on p.1624 of \citep{LelNadSch10}; this guarantees the weak convergence of $P_N$ to $\delta_{\theta_\infty}$, the Dirac distribution at $\theta_\infty$.
Therefore,
\begin{eqnarray}
&& \lim_{N \to \infty} \left\| \mu_{P_N} - k(\cdot,\theta_\infty) \right\|_{\mathcal{H}}^2 \nonumber \\
&=& \lim_{N \to \infty} \left< \mu_{P_N}, \mu_{P_N} \right>_{\mathcal{H}} - 2 \lim_{N \to \infty} \left< \mu_{P_N}, k(\cdot,\theta_\infty) \right>_{\mathcal{H}} \nonumber \\ && +  \left<k(\cdot,\theta_\infty), k(\cdot,\theta_\infty) \right>_{\mathcal{H}} \nonumber \\
&=& \lim_{N \to \infty} \int \int k(\theta,\theta') dP_N(\theta) dP_N(\theta') \nonumber \\
&& - 2 \lim_{N \to \infty} \int k(\theta,\theta_\infty)dP_N(\theta) + k(\theta_\infty,\theta_\infty) \nonumber \\
&=& k(\theta_\infty,\theta_\infty) - 2 k(\theta_\infty,\theta_\infty) + k(\theta_\infty,\theta_\infty) \label{eq:limit} \\
&=& 0, \nonumber
\end{eqnarray}
where (\ref{eq:limit}) follows from the weak convergence of $P_N$ to $\delta_{\theta_\infty}$ and $k$ is continuous and bounded, and we use Theorem 2.8 (ii) in \cite{Bil99} for the first term in (\ref{eq:limit}).
\end{proof}
\end{comment}

\begin{comment}
\begin{corollary}
\label{cor:consistency}
\begin{eqnarray}
\operatornamewithlimits{argmin}_{x}  \lim_{N \to \infty} \left\lVert \mu_{p_N} - k(\cdot,x)  \right\rVert_{\mathcal{H}}^2 = \theta_{\infty}.
\end{eqnarray}

\begin{proof}
\begin{eqnarray}
&& \operatornamewithlimits{argmin}_{x}  \lim_{N \to \infty} \left\lVert \mu_{p_N} - k(\cdot,x)  \right\rVert_{\mathcal{H}}^2 \\
&=& \operatornamewithlimits{argmin}_{x} \lim_{N \to \infty} -2 \int k(x,\theta)dP_{N}(\theta) \\
&=& \operatornamewithlimits{argmax}_{x} \int k(x,\theta)d\delta_{\theta_{\infty}}(\theta)\\
&=& \operatornamewithlimits{argmax}_{x} k(x,\theta_{\infty})\\
&=& \theta_{\infty}
\end{eqnarray}
\end{proof}
\end{corollary}
\begin{remark}
Corollary \ref{cor:consistency} states that the first sample from the power posterior $P_N$ via kernel herding is consistent for MLE as $N \to \infty$.
\end{remark}
\end{comment}

\begin{proposition}
\label{pr:consistency2}
Let $\Theta \subset \mathbb{R}^d$ be a compact set, and $k: \Theta \times \Theta \to \mathbb{R}$ be a continuous, bounded kernel.
Let $\theta_N := \operatornamewithlimits{argmin}_{\tilde{\theta} \in \Theta} \left\lVert \mu_{P_N} - k(\cdot,\tilde{\theta}) \right\rVert_{\mathcal{H}}$.
If Assumptions \ref{as:likelihood_prior} and \ref{as:kernel} hold, then we have $\theta_N \to \theta_{\infty}$ as $N \to \infty$.
\end{proposition}

\begin{comment}

\begin{proof}
First note that $\theta_N$ exists because $\Theta$ is compact and $k$ is continuous.
By the reproducing property and Assumption \ref{as:kernel}, we have
\begin{eqnarray*}
\theta_N &=& \operatornamewithlimits{argmin}_{\tilde{\theta} \in \Theta} \left\lVert \mu_{P_N} - k(\cdot,\tilde{\theta})  \right\rVert_{\mathcal{H}}^2 \\
&=& \operatornamewithlimits{argmin}_{\tilde{\theta} \in \Theta} \left\| \mu_{P_N} \right\|_{\mathcal{H}}^2 \nonumber -2 \mu_{P_N}(\tilde{\theta}) + k(\tilde{\theta},\tilde{\theta}) \\
&=& \operatornamewithlimits{argmax}_{\tilde{\theta} \in \Theta} \mu_{P_N}(\tilde{\theta}).
\end{eqnarray*}
By the reproducing property, the Cauchy-Schwartz inequality and Assumption \ref{as:kernel}, we have for all $\theta \in \Theta$
\begin{eqnarray}
\left| \mu_{P_N}(\theta) -  k(\theta,\theta_\infty) \right|
&=& \left| \left< k(\cdot,\theta), \mu_{P_N} -  k(\cdot,\theta_\infty) \right> \right| \nonumber \\
&\leq& \sqrt{ k(\theta,\theta) } \left\| \mu_{P_N} -  k(\cdot,\theta_\infty) \right\|_{\mathcal{H}} \nonumber \\
&=& \sqrt{ C } \left\| \mu_{P_N} -  k(\cdot,\theta_\infty) \right\|_{\mathcal{H}} \label{eq:uniform}
\end{eqnarray}

Let $\varepsilon$ be an arbitrary positive number and $U_\varepsilon(\theta_\infty)$ be an open $\varepsilon$-neighborhood of $\theta_\infty$.  From Assumption \ref{as:kernel} (ii) and continuity of $k$, there is $\delta>0$ such that
\begin{equation}
\max_{\theta\in\Theta\backslash U_\varepsilon(\theta_\infty)} k(\theta,\theta_\infty) \leq C-\delta. \label{eq:outof}
\end{equation}
It follows from Eq.(\ref{eq:uniform}) and Proposition \ref{prop:convergence} that there is $N_0\in\mathbb{N}$ such that
\begin{equation}
\max_{\theta\in\Theta} \left| \mu_{P_N}(\theta) -  k(\theta,\theta_\infty) \right| \leq \delta/3 \label{eq:unifbound}
\end{equation}
holds for all $N\geq N_0$.
This implies, in particular, that for all $N\geq N_0$
\begin{equation}
\mu_{P_N}(\theta_\infty) \geq k(\theta_\infty,\theta_\infty) -\delta/3 = C-\delta/3.  \label{eq:lbpn}
\end{equation}
On the other hand, using Eqs.(\ref{eq:outof}) and (\ref{eq:unifbound}), we have
\begin{equation}
\max_{\theta\in\Theta\backslash U_\varepsilon(\theta_\infty)} \mu_{P_N}(\theta)
\leq C-\frac{2}{3}\delta \label{eq:outof2}
\end{equation}
for all $N\geq N_0$.

Eqs.(\ref{eq:lbpn}) and (\ref{eq:outof2}) show that the maximum of $\mu_{P_N}$ is attained in $U_\varepsilon(\theta_\infty)$, that is, $\theta_N \in U_\varepsilon(\theta_\infty)$, for all $N\geq N_0$, which completes the proof.
\end{proof}
\end{comment}

\begin{comment}
\begin{proof}
First note that $\theta_N$ exists because $\Theta$ is compact and $k$ is continuous.
By the reproducing property and Assumption \ref{as:kernel}, we have
\begin{eqnarray*}
\theta_N &=& \operatornamewithlimits{argmin}_{\tilde{\theta} \in \Theta} \left\lVert \mu_{P_N} - k(\cdot,\tilde{\theta})  \right\rVert_{\mathcal{H}}^2 \\
&=& \operatornamewithlimits{argmin}_{\tilde{\theta} \in \Theta} \left\| \mu_{P_N} \right\|_{\mathcal{H}}^2 \nonumber -2 \mu_{P_N}(\tilde{\theta}) + k(\tilde{\theta},\tilde{\theta}) \\
&=& \operatornamewithlimits{argmax}_{\tilde{\theta} \in \Theta} \mu_{P_N}(\tilde{\theta}).
\end{eqnarray*}
By the reproducing property, the Cauchy-Schwartz inequality, and Assumption \ref{as:kernel}, we have for all $\theta \in \Theta$
\begin{eqnarray}
\left| \mu_{P_N}(\theta) -  k(\theta,\theta_\infty) \right|
&=& \left| \left< k(\cdot,\theta), \mu_{P_N} -  k(\cdot,\theta_\infty) \right> \right| \nonumber \\
&\leq& \sqrt{ k(\theta,\theta) } \left\| \mu_{P_N} -  k(\cdot,\theta_\infty) \right\|_{\mathcal{H}} \nonumber \\
&=& \sqrt{ C } \left\| \mu_{P_N} -  k(\cdot,\theta_\infty) \right\|_{\mathcal{H}} \label{eq:uniform}
\end{eqnarray}
We first show that $\lim_{N \to \infty} k(\theta_N,\theta_\infty) = k(\theta_\infty,\theta_\infty)$.
Let $\varepsilon > 0$ be arbitrary.
Then, by Proposition \ref{prop:convergence}, there exists an $N(\varepsilon) \in \mathbb{N}$ such that $\sqrt{ C } \left\| \mu_{P_N} -  k(\cdot,\theta_\infty) \right\|_{\mathcal{H}} \leq \varepsilon$ for all $N > N(\varepsilon)$.
Let $N > N(\varepsilon)$ be fixed.
By Eq.(\ref{eq:uniform}), we then have
$$ \left| \mu_{P_N}(\theta) -  k(\theta,\theta_\infty) \right|  \leq \varepsilon, \quad \forall \theta \in \Theta.$$
From this inequality and the fact that $\theta_N$ and $\theta_\infty$ are, respectively, the maximizers of $\mu_{P_N}$ and $k(\cdot,\theta_\infty)$, one can show that $| \mu_{P_N} (\theta_N) - k(\theta_\infty,\theta_\infty) | \leq \varepsilon$ as follows:
(i) if $\mu_{P_N} (\theta_N) \geq k(\theta_\infty,\theta_\infty)$, we have $\mu_{P_N} (\theta_N) \geq k(\theta_\infty,\theta_\infty) \geq k(\theta_N,\theta_\infty)$ and thus $0 \leq \mu_{P_N} (\theta_N) - k(\theta_\infty,\theta_\infty) \leq \mu_{P_N} (\theta_N) - k(\theta_N,\theta_\infty) \leq \varepsilon$;
(ii) if $k(\theta_\infty,\theta_\infty) > \mu_{P_N}(\theta_N)$, then $k(\theta_\infty,\theta_\infty) > \mu_{P_N}(\theta_N) \geq \mu_{P_N}(\theta_\infty)$, and thus $0 < k(\theta_\infty,\theta_\infty) - \mu_{P_N} (\theta_N) \leq  k(\theta_\infty,\theta_\infty) - \mu_{P_N} (\theta_\infty) \leq \varepsilon$.
Therefore, the triangle inequality yields
$| k(\theta_N,\theta_\infty) - k(\theta_\infty,\theta_\infty) |
\leq | k(\theta_N,\theta_\infty) - \mu_{P_N} (\theta_N) | + | \mu_{P_N} (\theta_N) - k(\theta_\infty,\theta_\infty) | \leq 2 \varepsilon$.
This results in the conclusion that $\lim_{N \to \infty} k(\theta_N,\theta_\infty) = k(\theta_\infty,\theta_\infty)$.

We now show the assertion $\theta_N \to \theta_{\infty}$ via proof by contradiction.
Assume that $\theta_N$ does not converge to $\theta_\infty$ as $N \to \infty$.
Then there exist a constant $\delta > 0$ and a subsequence $\{\theta_{N_\ell} \}_{\ell=1}^\infty$ with $N_1 < N_2 < \cdots$ such that $\| \theta_{N_\ell} - \theta_\infty \| > \delta$ holds for all $\ell \in \mathbb{N}$.
Since $\Theta$ is compact, there exists a convergent subsequence of $\{\theta_{N_\ell} \}_{\ell=1}^\infty$, such as $\{\theta_{ N_{\ell_i} } \}_{i=1}^\infty$ with $\ell_1 < \ell_2 < \cdots$.
Let $\theta_* \in \Theta$ be the limit of $\{\theta_{ N_{\ell_i} } \}_{i=1}^\infty$ as $i \to \infty$.
Then we have $\| \theta_* - \theta_\infty \| \geq \delta$, i.e., $\theta_* \neq \theta_\infty$.
On the other hand, since $k$ is continuous, we have $\lim_{i=1}^\infty k(\theta_{ N_{\ell_i} }, \theta_\infty) = k(\theta_*, \theta_\infty) = k(\theta_\infty, \theta_\infty)$, where the second equality holds from the argument in the previous paragraph.
This is a contradiction with Assumption \ref{as:kernel}, which implies that $k(\theta, \theta_\infty) < k(\theta_\infty, \theta_\infty)$ for all $\theta \neq \theta_\infty$.
\end{proof}
\end{comment}

\begin{comment}
\begin{remark}
For simplicity, we assume in Proposition \ref{pr:consistency2} that $\theta$ is compact, but this condition can be relaxed.
For example, we may instead assume the following weaker condition:
For any open neighborhood $U$ of $\theta_\infty$, there is a positive constant $\delta$ such that
$\sup_{\theta\in \Theta\backslash U} k(\theta,\theta_\infty) \leq k(\theta_\infty,\theta_\infty) - \delta$.
\end{remark}
\end{comment}

\begin{comment}

\begin{proposition}
\label{pr:consistency2}
Let $\Theta \subset \mathbb{R}^d$ be a Borel measurable compact set, $k: \Theta \times \Theta \to \mathbb{R}$ be a continuous, bounded kernel and
\begin{eqnarray}
\theta_N := \operatornamewithlimits{argmin}_{\tilde{\theta}} \left\lVert \mu_{P_N} - k(\cdot,\tilde{\theta}) \right\rVert_{\mathcal{H}}^2.
\end{eqnarray}

where $\theta_N \in \Theta$ is the first sample by kernel herding. Then $\theta_N \to \theta_{\infty}$ as $N \to \infty$.
\end{proposition}

\begin{proof}
\begin{eqnarray}
\theta_N &:=& \operatornamewithlimits{argmin}_{\tilde{\theta}} \left\lVert \mu_{P_N} - k(\cdot,\tilde{\theta})  \right\rVert_{\mathcal{H}}^2 \\
&=& \operatornamewithlimits{argmin}_{\tilde{\theta}} \left< \mu_{P_N}, \mu_{P_N}\right>_{\mathcal{H}} \nonumber \\
&&-2 \int k(\tilde{\theta},\theta)dP_{N}(\theta) +
\left< k(\cdot,\tilde{\theta}), k(\cdot,\tilde{\theta})\right>_{\mathcal{H}}\\
&=& \operatornamewithlimits{argmax}_{\tilde{\theta}} \int k(\tilde{\theta},\theta)dP_{N}(\theta)\\
&=& \operatornamewithlimits{argmax}_{\tilde{\theta}} \mu_{P_N}(\tilde{\theta})
\end{eqnarray}

Eq. 20 follows from the fact that the first and third terms in Eq.19 are constant because of Assumption 2.

Then there exists a subsequence $\{\theta_{N(l)}\}_{l=1}^{\infty}$ such that $\theta_{N(l)} \to \hat{\theta}$. Similarly, we can take a subsequence $\{\mu_{P_N(l)}\}_{l=1}^{\infty}$. This is immediate from Proposition \ref{prop:convergence}, $\mu_{P_N(l)} \to k(\cdot, \theta_{\infty})$. From the definition of $\operatornamewithlimits{argmax}$, $\mu_{P_N(l)}(\theta_{N(l)}) \geq \mu_{P_N(l)} (\theta)$ for all $\theta \in \Theta$. Then, letting $l \to \infty$, we get $k(\hat{\theta}, \theta_{\infty}) \geq k(\theta, \theta_{\infty})$ for all $\theta \in \Theta$. From Assumption 2, $k(x,\theta_{\infty})$ takes the maximum at $x = \theta_{\infty}$.
Therefore, $\hat{\theta} = \theta_{\infty}$. This convergence holds for any subsequence of $\{\theta_{N}\}_{N=1}^{\infty}$. The convergence of any subsequence implies the convergence of the whole sequence $\{\theta_{N}\}_{N=1}^{\infty}$. Since a convergent subsequence $\{\theta_{N(l)}\}_{l=1}^{\infty}$ has the same limit as $\{\theta_{N}\}_{N=1}^{\infty}$, $\theta_{N} \to \theta{\infty}$.
\end{proof}
\end{comment}

We make a few remarks regarding Assumption \ref{as:likelihood_prior}.
The assumption that $\ell$ has a unique global maximum is not satisfied if the model is singular, an example being mixture models: In this case there are multiple global maximums.
However, our experiment in Sec.~\ref{sec:gaussian-mixture} shows that even for mixture models, the proposed method works reasonably well.
This suggests that, in an empirical setting, a point estimate may converge to one of the global maximums.
The assumption $\pi(\theta_\infty) > 0$ will also not be satisfied if the support $\pi$ does not contain $\theta_\infty$, but the proposed method performs well even in this case (as shown in \ref{sec:Gauss-misspecified}), possibly thanks to the auto-correction mechanism explained above.
We reserve further analysis of these properties for future work.

\section{Experiments} \label{sec:experiments}
We have conducted a variety of experiments comparing the proposed method with existing approaches.
We begin with a quick review of these approaches (Sec.~\ref{sec:existing-methods}), and report experimental results on point estimation with a misspecified prior (Sec.~\ref{sec:Gauss-misspecified}), population dynamics of the blowfly (Sec.~\ref{sec:blowfly}), alpha stable distributions (Sec.~\ref{sec:alpha-stable}), Gaussian mixture models with redundant components (Sec.~\ref{sec:gaussian-mixture}), and a real-world pedestrian simulator (Sec.~\ref{sec:pedestrian-sim}).

\subsection{Existing approaches and experimental settings} \label{sec:existing-methods}
{\bf K2-ABC} \citep{pmlr-v51-park16} is an ABC method that represents the empirical distributions of simulated and test observations as kernel means in an RKHS.
For each of simulated parameters, the associated weight is calculated by using the RKHS distance between the kernel means (i.e., MMD), and the resulting weighted sample is treated as a posterior distribution.
{\bf Adaptive SMC-ABC} \citep{del2012adaptive} is a rejection-based approach based on sequential Monte Carlo, which sequentially updates the tolerance level and the associated proposal distribution in an adaptive manner.
This method is a state-of-the-art ABC approach.
The approach by \citet{gutmann2016bayesian}, which we refer to as {\bf Bayesian Optimization} for simplicity, is a method for MLE with intractable likelihood based on Bayesian optimization \citep{brochu2010tutorial}.
This method optimizes the parameters in a intractable model so as to minimize the discrepancy between the simulated and test observations.
Note that comparison with this method in terms of computation time may not make sense (although we report them for purposes of completeness), as we used publicly available code\footnote{https://sheffieldml.github.io/GPyOpt/} for implementation.
{\bf The method of simulated moments (MSM)} \citep{mcfadden1989method} optimizes the parameter in the model so that the resulting moments of simulated data match those of observe data.
MSM may be seen a special case of indirect inference \citep{gourieroux1993indirect}, an approach studied in econometrics.\footnote{MSM is a special case of indirect inference because the moments can be regarded as the parameters of an auxiliary model.}
{\bf Data-cloning ABC (ABC-DC)} \citep{picchini2017approximate} is an approach combining ABC-MCMC \citep{marjoram2003markov} and Data Cloning \citep{LelNadSch10}, replicating observed data multiple times to achieve MLE with intractable likelihood.

\vspace{-0.5cm}
\paragraph{Experimental settings.}
Unless otherwise specified, the following settings were applied in the experiments.
For all the methods that employed kernels, we used Gaussian kernels.
The discrepancy between the simulated and observed data was measured by the {\em energy distance} \citep{szekely2013energy}, which is a standard metric for distributions in statistics and can be computed only from pairwise Euclidean distances between data points.
Since the usual quadratic time estimator was too costly, we used a linear time estimator for computing the energy distance (see the Supplementary Materials for details).

For each method, unless otherwise specified, we determined the hyper-parameters on the basis of the cross-validation-like approach described in \citet[Sec.~4]{pmlr-v51-park16}.
That is, to evaluate one configuration of hyper-parameters, we first used 75\% of the observed data for point estimation and then computed the discrepancy between the rest of the observed data and the ones simulated from point estimates; after applying this procedure to all candidate configurations, the one with the lowest discrepancy was finally selected.
The bandwidth of a Gaussian kernel was selected from candidate values, each of which is the median (of pairwise distances) multiplied by logarithmically equally spaced values between $2^{-4}$ and $2^4$ \citep[Sec.~5.1.1]{takeuchi2006nonparametric}.
Regularization constants for the proposed method and kernel ABC, as well as the soft threshold for K2-ABC, were selected from logarithmically spaced values between $10^{-4}$ and $1$.
To compute MMD for K2-ABC, a linear time estimator \citep[Sec.~6]{gretton2012kernel} was used to reduce computational time, as the usual quadratic time estimator was too costly.
For Adaptive SMC-ABC, the initial tolerance level was set as the median of pairwise distances between the observed and simulated data.
For Bayesian Optimization, we used Expected Improvement as an acquisition function, and all the hyper-parameters were marginalized out following the approach of \citet[Sec.~3.2]{snoek2012practical}.
For MSM, the number of moments were selected from a range up to 30 by the cross-validation like approach.
For ABC-DC, we employed, in particular, dynamic ABC-DC, which automatically adjusts its associated parameters.
To obtain point estimates with kernel ABC and K2-ABC, we computed the means of the resulting posterior distributions.
For Adaptive SMC-ABC, point estimates were obtained as posterior means as well as MAP estimates by applying the mean shift algorithm to posterior weighted samples \citep{fukunaga1975estimation}, the latter essentially being an approach suggested by \citet{rubio2013simple}.

The following abbreviations may be used for the sake of simplicity; kernel recursive ABC is referred to as {\bf KR-ABC}, kernel ABC as {\bf K-ABC}, adaptive SMC-ABC as {\bf SMC-ABC}, Bayesian Optimization as {\bf BO}, and Dynamic ABC-DC as {\bf ABC-DC}.
For our method, we also report results based on a half number of iterations, which we call {\bf KR-ABC (less)}.

\subsection{Multivariate Gaussian distribution with a severely misspecified prior} \label{sec:Gauss-misspecified}
As a proof of concept regarding the auto-correction mechanism of the proposed method described in Sec.\ref{sec:proposed-algorithm}, we have performed an experiment for when the prior distribution is severely misspecified (see the Supplementary Materials for an illustration).
The task is to estimate the mean vector of a 20-dimensional Gaussian distribution ${\rm Normal}(\mu,\Sigma)$, where the true mean vector is $\mu := (10,50,90,130,180,280,390,430,
520,630,1010,1050,\\ 1090,1130,1180,1280,1390,1430,1520,1630)^T \in \mathbb{R}^{20}$.
The covariance matrix $\Sigma \in \mathbb{R}^{20 \times 20}$ is assumed to be known and is a diagonal matrix with all diagonal elements being $40$.
Test data $y^*$ consisted of 100 i.i.d.~observations from this Gaussian distribution.
As a prior for the mean vector $\mu$, we used the uniform distribution on $[9\times 10^6, 10^7]^{20}$, which is extremely misspecified.
For Bayesian optimization, the space to be explored was set as $[0, 10^7]^{20}$.

In this experiment, each pseudo data was made from 100 observations simulated with one parameter configuration.
K2-ABC and K-ABC used 3000 pairs of a parameter and pseudo-data.
For the proposed method and SMC-ABC, we generated 100 pairs of a parameter and pseudo-data for the initial iteration, and then the iterations were repeated 30 times, resulting in a total of 3000 simulations.
For the proposed method, the bandwidth of the kernel $k_{{\mathcal{Y}}}$ on observed data was recomputed for each iteration, using the median heuristic.
For SMC-ABC, the parameter $\alpha \in (0,1)$, which controls the trade-off between the speed of convergence and the accuracy of posterior approximation, was set to be 0.3, as we found this value to be the best in terms of the trade-off.

For each method, we ran 30 independent trials, and the results in averages and standard deviations are shown in Table \ref{tab:table_mvincorrect}, where the {\em parameter error} is the mean (over 20 dimensions) of the absolute difference between the estimated and the true parameter values divided by the true value, and the {\em data error} is the energy distance between the true data and pseudo data simulated with the estimated parameter.
Surprisingly, the proposed method successfully approached the true parameter even when the prior was severely misspecified.
As discussed in Sec~\ref{sec:proposed-algorithm} and demonstrated in the Supplementary Materials, this would appear to be because of the use of kernel herding, which automatically widens the space to explore when simulated data is far apart from test data.
As expected, other methods were unable to approach the true parameter.

\begin{comment}
\begin{table}[t]
  \caption{Results for the pedestrian simulator in Sec.~\ref{sec:pedestrian-sim}} \label{tab:table3}
  \begin{center}
  \begin{adjustbox}{max width=230pt}
  \begin{tabular}{|l|c|r|r|r|r|} \hline
    Algorithm & $\theta^{(N)}$ error & $\theta^{(T)}$ error & data error& cputime \\ \hline
    KR-ABC & 61.58(74.42) & 70.93(102.08) & 0.008(0.009) & 2233.45(97.54)\\ \hline
    KR-ABC (less) & 82.46(75.05) & 134.00(161.85) & 0.014(0.014) & 1875.32(147.16)\\ \hline
    K2-ABC & 298.94(120.71) & 308.95(109.43) & 0.10(0.10)& 1547.32(56.31)\\ \hline
    K-ABC & 354.72(145.76) & 389.52(140.91) & 0.12(0.09)& 1773.74(84.91)\\ \hline
    SMC-ABC (mean) & 271.51(104.64) & 363.12(91.28) & 0.09(0.07)& 2017.89(110.02)\\ \hline
    SMC-ABC (MAP) & 255.15(139.33) & 348.43(104.74) & 0.09(0.1)& 2017.89(110.02)\\ \hline
    ABC-DC & 273.93(136.14)& 327.48(98.12) & 0.09(0.14) & 1984.43(59.12)\\ \hline
    BO & 194.57(65.83) & 291.73(105.33) & 0.04(0.06) & 37541.23(3047.46)\\ \hline
    MSM & 453.58(89.43) & 510.04(55.10) & 0.24(0.17) & 1869.83(49.51)\\ \hline

  \end{tabular}
  \end{adjustbox}
  \end{center}
  \vspace{-0.25in}
\end{table}
\end{comment}

\begin{table}[t]
  \caption{Results for multivariate Gaussian distributions in Sec.~\ref{sec:Gauss-misspecified}}
    \begin{center}
   \begin{adjustbox}{max width=230pt,
   max height = 100pt}
  \begin{tabular}{|l|c|r|r|r|} \hline
    Algorithm & parameter error & data error& cputime \\ \hline
    KR-ABC & 0.70(0.29) & 0.008(0.004) & 866.02(26.12) \\ \hline
    KR-ABC (less) & 7.22(3.28) & 0.02(0.24) & 353.498(23.05) \\ \hline
    K2-ABC & >1e+6 (>1e+3) & >1e+5 (>1e+3) & 209.51(11.49) \\ \hline
    K-ABC & >1e+6 (>1e+3) &>1e+5 (>1e+3)) & 403.93(24.97) \\ \hline
    SMC-ABC (mean) & >1e+6 (>1e+3) & >1e+5 (>1e+3) & 590.41(29.54)\\ \hline
	SMC-ABC (MAP) & >1e+6 (>1e+3) & >1e+5 (>1e+3) & 590.41(29.54)\\ \hline
    ABC-DC & >1e+6 (>1e+3) & >1e+5 (>1e+3) & 313.99(16.85)\\ \hline
    BO & >1e+5(>1e+4) & >1e+5 (>1e+4)&  25940.86(936.40)\\ \hline
    MSM & >1e+5(>1e+4) & >1e+5(>1e+4)& 307.42(67.94)\\ \hline
  \end{tabular}
  \end{adjustbox}
  \label{tab:table_mvincorrect}
  \end{center}
    \vspace{-0.15in}
\end{table}

\begin{comment}

\begin{table}[h]
  \caption{Error on the experiment of multivariate Gaussian distribution.}
    \begin{center}
   \begin{adjustbox}{max width=230pt}
  \begin{tabular}{|l|c|r|r|r|} \hline
    Algorithm & parameter error & data error& cputime \\ \hline
    KR-ABC & 0.007(0.003) & 0.008(0.004) & 866.02(26.12) \\ \hline
    KR-ABC (less) & 0.04(0.32) & 0.02(0.24) & 353.498(23.05) \\ \hline
    K2-ABC & >1e+5 (>1e+3) & >1e+5 (>1e+3) & 209.51(11.49) \\ \hline
    K-ABC & >1e+5 (>1e+3) &>1e+5 (>1e+3)) & 403.93(24.97) \\ \hline
    SMC-ABC (mean) & >1e+5 (>1e+3) & >1e+5 (>1e+3) & 590.41(29.54)\\ \hline
	SMC-ABC (MAP) & >1e+5 (>1e+3) & >1e+5 (>1e+3) & 590.41(29.54)\\ \hline
    ABC-DC & >1e+5 (>1e+3) & >1e+5 (>1e+3) & 313.99(16.85)\\ \hline
    BO & 2229.66(779.20) & >1e+5 (>1e+3)&  25940.86(936.40)\\ \hline
    MSM & 0.913(0.12) & 900.37(3.47)& 508.03(48.13)\\ \hline
  \end{tabular}
  \end{adjustbox}
  \label{tab:table_mvincorrect}
  \end{center}
\end{table}

\begin{figure}[t]
\vspace{.3in}
\begin{minipage}{1.0\hsize}
\begin{center}
\includegraphics[scale = 0.3]{incorrect_prior.png}
\end{center}
\end{minipage}
\vspace{.3in}
\caption{Experiment on multivariate Gaussian distribution.}
\label{fig:mv_incorrect}
\end{figure}
\end{comment}

\subsection{Ecological dynamic systems: blowfly}
\label{sec:blowfly}
Following \citet{pmlr-v51-park16}, we performed an experiment on parameter estimation with a dynamical system of blowfly populations \citep{Woo10}, which is defined as
\begin{equation*}
N_{t+1} = PN_{t-\tau} \exp\left(- N_{t-\tau} / N_{0} \right)\it{e_{t}} + \it{N_{t}} \exp(-\delta \epsilon_{t}),
\end{equation*}
where $t = 1,\dots,T$ are time indices, $N_t$ is the population at time $t$, $e_{t} \sim {\rm Gam}(\frac{1}{\sigma_{p}^{2}} \sigma_{p}^{2})$ and $\epsilon_{t} \sim {\rm Gam}(\frac{1}{\sigma_{d}^{2}}, \sigma_{d}^{2})$ are independent Gamma-distributed noise, and $\theta := (P \in \mathbb{N},\ N_{0} \in \mathbb{N},\ \sigma_{d} \in \mathbb{R}_{+},\ \sigma_{p} \in \mathbb{R}_{+},\ \tau \in \mathbb{N},\ \delta\ \in \mathbb{R}_{+})$ are the parameters of the system.
The task is to estimate $\theta$ from observed values of $N_1,\dots,N_T$.
We set the true parameters as $\theta = (29, 260, 0.6, 0.3, 7, 0.2)$, and the time-length $T$ for both the observed and pseudo data as $T = 1000$. Following \citet[Sec.~4]{pmlr-v51-park16}, for each parameter we defined a Gaussian prior on its logarithm (see the Supplementary Materials for a definition). In this experiment, for all methods we converted the observed and pseudo-data into histograms with 1000 bins (i.e., we treated each data as a 1000 dim.~vector), as this produced better results.
K2-ABC and K-ABC used 1300 pairs of a parameter and a pseudo-data item.
For the proposed method and SMC-ABC, we generated 100 pairs of a parameter and pseudo-data for the initial iteration, and the iterations were then repeated 13 times, resulting in total 1300 simulations.
For SMC-ABC, we set the parameter $\alpha \in (0,1)$ to be 0.3, as in Sec.~\ref{sec:Gauss-misspecified}.

For each method we performed 30 independent trials, and the results are summarized in Table \ref{tab:table1}.
The proposed method performed the best, even when the number of simulations was halved (i.e., KR-ABC (less)).

\begin{table}[t]
  \caption{Results for blowfly population dynamics in Sec.\ref{sec:blowfly}}
    \begin{center}
    \begin{adjustbox}{max width=210pt}
  \begin{tabular}{|l|c|r|r|r|} \hline
    Algorithm & parameter error & data error & cputime\\ \hline
    KR-ABC & 0.47(0.11) & 43.85(37.24) &  101.143(13.25) \\ \hline
    KR-ABC (less) & 0.57(0.21) & 67.57(47.11) &  32.98(1.21) \\ \hline
    K2-ABC & 0.81(0.42) & 67.45(77.86) &  23.47(1.59) \\ \hline
    K-ABC & 0.62(0.09) & 89.37(29.22) &  30.66(2.57)  \\ \hline
    SMC-ABC (mean) & 0.83 (0.10) & 170.41 (47.91) &  38.50(2.34) \\ \hline
	SMC-ABC (MAP) & 0.84 (0.12) & 163.19(42.51) &  38.50(2.34) \\ \hline
    ABC-DC & 0.89(0.17) & 134.12(58.92) &  29.94(4.57)\\ \hline
    BO &0.70(0.28) & 108.18(67.08) & 3217.40(157.31)\\ \hline
    MSM & 0.67(0.08) & 89.17(33.20) &  25.46(8.26)\\ \hline
  \end{tabular}
  \end{adjustbox}
  \label{tab:table1}
  \end{center}
  \vspace{-0.15in}
\end{table}

\subsection{Multivariate elliptically contoured alpha stable distribution}
\label{sec:alpha-stable}
In addition to the competitive methods described earlier, we performed a comparison with the method called {\em noisy ABC-MLE} \citep{yildirim2015parameter}.
This method assumes that sampling from the intractable model can be realized by deterministic mapping applied to a simple random variable, and that the gradient of the deterministic mapping is available.
This method can, then, only be applied to a limited class of generative models, though for such models it can perform well.
Although the main scope of this paper is on simulation models in which such gradient information is unavailable, we performed this experiment in order to see how the proposed method compared with this method without relying on the gradient information.

We also considered parameter estimation with {\em multivariate elliptically contoured alpha stable distributions} \citep{nolan2013multivariate}, which subsume heavy-tailed and skewed distributions and are popular for modeling financial data.
This family of distributions in general does not admit closed-form expressions for density functions, which means they are ``intractable'' in the sense that the standard procedure for parameter estimation cannot be employed.
However, sampling of a random vector ${\bm X} \in \mathbb{R}^d$ from this family is possible in the following way:
\begin{eqnarray*}
{\bm X}& :=& A^{1/2} {\bm G} + \delta \in \mathbb{R}^d,\quad
{\bm G} \sim  {\rm Normal}({\bm 0},Q), \\
A& :=& \tau_{\theta}(U_1,U_2) \in \mathbb{R},\\
&& U_1  \sim   {\rm Unif}(-\pi/2, \pi/2),\quad U_{2} \sim {\rm Exp}(1),
\end{eqnarray*}
where  $Q \in \mathbb{R}^{d \times d}$ is positive definite,  $\delta \in \mathbb{R}^{d}$, $\theta := (\alpha, \beta, \mu, \sigma) \in (0,2] \times [-1,1] \times \mathbb{R} \times [0,\infty)$,  $\tau_{\theta}$ is a deterministic mapping whose concrete form is described in the Supplementary Materials, and ${\rm Unif}$ and ${\rm Exp}$ denote uniform and exponential distributions, respectively.

We dealt with estimation of $\alpha := 1.3$ and $Q$, while fixing the other parameters as $\delta := {\bm 0}$, $\beta := 1$, $\mu := 0$, and $\sigma := 1$.
We restricted $Q$ to be a positive definite matrix such that all diagonal elements are the same and so are the off-diagonal elements.
We defined true $Q$ to be a matrix whose diagonal elements are $1.0$ and off-diagonals are $0.2$.
Therefore the task was to estimate these three values (i.e., $1.3$ for $\alpha$, and $1.0$ and $0.2$ for $Q$).
We used ${\rm Unif}[0,2]$ as a prior for $\alpha$, and ${\rm Unif}[0,5]$ as a prior for each of the diagonal and off-diagonal values of $Q$.

Figure \ref{fig:alpha} shows results for the averages of mean square errors in parameter estimation over 30 independent trials, with variation in the dimensionality $d$ from $2$ to $16$.
For each method we sampled a total of 1400 pairs of a parameter and a pseudo-data item, and for iterative methods we used 100 pairs in each iteration.
Each pseudo-data (and observed-data) was made up of 1000 points.
The noisy ABC-MLE exploited the gradient information in $\tau_\theta$, while the other methods did not.
The proposed method was competitive with BO and outperformed the other methods with the exception of the noisy ABC-MLE.
Although the noisy ABC-MLE was accurate for lower-dimensionality (as expected), it exhibited a steep increase in errors for higher dimensionality. In contrast, the performance degradation of the proposed method was mild for higher dimensionality.

\begin{figure}[t]
\begin{minipage}{0.48\hsize}
\begin{center}
\includegraphics[scale = 0.12]
{output_45_0.png}
\end{center}
\end{minipage}
\caption{Results for multivariate elliptically contoured alpha stable distributions in Sec.~\ref{sec:alpha-stable}.
For each dimension (vertical axis), the averages of mean squared errors (MSE) over 30 trials are shown.}
\vspace{-0.15in}
\label{fig:alpha}
\end{figure}

\begin{comment}
The difficult part to deal with is the unavailability of closed-form expressions of the density. Rather, its characteristic function is given by
\begin{eqnarray*}
\mathbb{E}[\exp(\sqrt{-1}{\bf u}^T {\bf X})] = \exp (-({\bf u}^{T} Q
{\bf u})^{\alpha/2} + \sqrt{-1} {\bf u}^{T}\delta).
\end{eqnarray*}

for a positive semi-definite matrix $Q \in \mathbb{R}^{d \times d} $ and shift vector $\delta \in \mathbb{R}^{d}$.
\end{comment}

\subsection{Gaussian mixture with redundant components}
\label{sec:gaussian-mixture}
We consider here a parametric model in which there exist redundant parameters for expressing given data.
We are interested in whether point estimation with the proposed method results in elimination of the redundant parameters when applied to such a model.
This was motivated by \citet{Yamazaki2013}, who argued that, for mixture models, the use of a Dirichlet prior with a sufficiently small concentration parameter leads to elimination of unnecessary components.
We therefore focus on mixture models with redundant components.

Specifically, we considered Gaussian mixture models.
We defined the true model as a {\em two-component} Gaussian mixture
$\sum_{i=1}^{2} \phi_i \rm{Normal}(\mu_i, 20)$ of equal variances.
The task was to estimate the mixture coefficients $(\phi_1,\phi_2,) := (0.7, 0.3)$ and the associate means $(\mu_1,\mu_2) := (110, 70)$, provided 3000 i.i.d.~sample points from the model as observed data $y^*$.
We employed an over-parametrized model for point estimation (i.e., no method used the knowledge that the truth consisted of 2 components), which is a {\em four-component} Gaussian mixture $\sum_{i=1}^{4} \phi_i \rm{Normal}(\mu_i, 20)$.
We used a 4-dimensional Dirichlet distribution with equal concentration parameters $0.01$ as a prior for the coefficients $(\phi_1,\dots,\phi_4)$, and ${\rm Normal} (0,100)$ as a prior for each of $\mu_1,\dots,\mu_4$.

For each method, we generated a total of 1000 pairs of a parameter and pseudo-data, and for iterative methods, we made use of 100 pairs in each iteration, resulting in 10 iterations.
Each pseudo-data consisted of 3000 simulated observations.
For all the methods, we converted each data item into a histogram of 300 bins and treated it as a 300 dim.~vector since this resulted in better performances.
We set the parameter $\alpha \in (0,1)$ of SMC-ABC to be 0.2, as this performed well in this experiment.

We ran each algorithm 30 times, and the resulting average errors and standard deviations are shown in Table \ref{tab:table2}, where the $\phi$ error and $\mu$ error denote the errors for the coefficients and the means, respectively, as measured in terms of Euclidean distance.
More precisely, since any permutation of component labels will result in the same model, we first sorted the estimated parameters $\{(\phi_i,\mu_i)\}$ so that $\phi_1 \geq \cdots \geq \phi_4$, and we then measured the errors w.r.t.~the ground truth $\phi := (0.7,0.3,0,0)$ and $\mu := (110,70)$.
For the $\mu$ error, we computed the errors only for the estimated means $\mu_1, \mu_2$ associated with the two largest coefficients since there was no ground truth for the redundant components $\mu_3, \mu_4$.
Results show that the proposed KR-ABC performed best, indicating that the dominant components were successfully estimated.

\begin{table}[t]
  \caption{Results for the Gaussian mixture model in Sec.~\ref{sec:gaussian-mixture}} \label{tab:table2}
   \begin{center}
   \begin{adjustbox}{max width=210pt,max height=40pt}
   \begin{tabular}{|l|c|r|r|r|r|} \hline
    Algorithm & $\phi$ error & $\mu$ error & data error & cputime \\ \hline
    KR-ABC & 0.159(0.106) & 54.14(8.71) & 0.03(0.05) & 281.59(12.55)\\ \hline
    KR-ABC (less) &0.22(0.13) & 64.04(17.58) & 0.11(0.15) & 147.39(16.76) \\ \hline
    K2-ABC & 0.53(0.02) & 93.98(3.84) & 0.69(0.10)& 89.43(7.56)\\ \hline
    K-ABC & 0.50(0.02) & 92.57(12.77) & 0.67(0.17)& 135.01(11.35)\\ \hline
    SMC-ABC (mean) & 0.51(0.04) & 83.19(27.86) & 0.23(0.15)& 214.35(8.77) \\ \hline
    SMC-ABC (MAP) & 0.21(0.13) & 72.76(56.24) & 0.12(0.08)& 214.35(8.77) \\ \hline
    ABC-DC & 0.48(0.16) & 137.84(50.06) & 0.43(0.14)& 149.74(21.22)\\ \hline
    BO & 0.37(0.08) & 80.79(36.17) & 0.82(0.68)& 13775.48(1438.9)\\ \hline
    MSM & 0.24(0.08) & 117.02(11.48) & 0.24(0.08)& 171.58(59.38)\\ \hline
  \end{tabular}
  \end{adjustbox}
  \end{center}
\vspace{-0.25in}
\end{table}

\subsection{Real-world pedestrian simulator}
\label{sec:pedestrian-sim}
Our final experiment was parameter estimation with {\em CrowdWalk}, a publicly available real-world simulator\footnote{https://github.com/crest-cassia/CrowdWalk} for the movements of pedestrians in a commercial district \citep{Noda2010p}.
It has been used to gain insights into pedestrian behavior at a variety of events and occurrences, such as fireworks festivals and evacuations after earthquakes.
As this simulator is complicated and also computationally expensive, its likelihood function is intractable.

Using CrowdWalk, we simulated the movements of pedestrians in Ginza, a commercial district in Tokyo (see Supplementary Materials for an illustration).
Specifically, we modeled pedestrians as a mixture of multiple groups,
each of which has the following 6 parameters (below $i$ denotes the index of a group):
(1) $\theta^{(N)}_i \in \mathbb{N}$: the number of pedestrians in the group;
(2) $\theta^{(T)}_i \in \mathbb{R}_{+}$: the time when the group starts to move;
(3) $\theta^{(S)}_i \in \mathbb{R}^2$: the starting location of the group (e.g., stations);
(4) $\theta^{(G)}_i \in \mathbb{R}^2$: the goal location of the group;
(5) $\theta^{(P)}_i \in \mathbb{R}^2$: the intermediate location(s) that the pedestrians in the group visit (e.g., stores); and
(6) $\theta^{(R)}_i \in \mathbb{R}_{+}$: the time duration(s) of the pedestrians' visit(s) at the intermediate location(s).

In this experiment, we focused on estimation of the first two parameters $\theta^{(N)}_i$, $\theta^{(T)}_i$, and fixed the other parameters.
We defined the true model as a mixture of 5 pedestrian groups, and set their parameters as $(\theta^{*(N)}_1,\dots,\theta^{*(N)}_5) := (100,100,100,100,100)$ and $(\theta^{*(T)}_1,\dots,\theta^{*(T)}_5) := (30, 60, 90, 120, 150)$.
As in Sec.~\ref{sec:gaussian-mixture}, we used a redundant model of a mixture of 10 groups for parameter estimation.
The goal was to detect the active 5 groups of the true model, without knowing that the truth consists of 5 groups.
For simplicity, 5 (unknown) groups among the 10 candidate groups included the parameters of the true model other than $\theta^{*(N)}_i$, $\theta^{*(T)}_i$; see the Supplementary Materials for details.

We defined prior distributions as follows.
First we assumed the total number $500$ of pedestrians to be known.
The mixing coefficients of the mixture of 10 groups are given by $(\phi_1,\dots,\phi_{10}) = (\theta^{(N)}_1,\dots,\theta^{(N)}_{10}) / 500$.
Thus, rather than directly putting a prior on $(\theta^{(N)}_1,\dots,\theta^{(N)}_{10})$, we defined a prior on the mixing coefficients $(\phi_1,\dots,\phi_{10})$.
Specifically, we used a Dirichlet prior with a small concentration parameter, as in Sec.~\ref{sec:gaussian-mixture}, in order to eliminate 5 redundant components:
\begin{eqnarray*}
(\phi_1,\dots,\phi_{10}) &\sim& {\rm Dirichlet}(\alpha_{1} ,..., \alpha_{10} ), \\
\theta^{(N)}_{i} &:=&  \phi_i * 500, \quad (i = 1,\dots, 10)
\end{eqnarray*}
where $\alpha_1 = \cdots = \alpha_{10} = 0.01$ denote the concentration parameters.
For each of $\theta^{(T)}_1,\dots,\theta^{(T)}_{10}$, we defined a broad uniform prior
$\theta^{(T)}_i \sim {\rm Unif}(0,480)$.

From the true model, we simulated 4200 time steps of pedestrian flow as observed data.
We made $5 \times 5 = 25$ grids in a map of Ginza and computed a histogram of the corresponding 25 bins for each time step.
Thus, observed data was made up of 4200 vectors in $\mathbb{R}^{25}$.
In the same way, each method generated a total of 4200 vectors, and each iterative method made use of 200 vectors in each iteration, running 21 iterations in total.
For SMC-ABC, we set the parameter $\alpha \in (0,1)$ to be 0.2, as in the previous experiment.

We ran each method 20 times, and the resulting averages and standard deviations for errors are summarized in Table \ref{tab:table3}, where ``$\theta^{(N)}$ error'' and ``$\theta^{(T)}$ error'' denote the errors of the corresponding estimated parameters, as measured in terms of Euclidean distance.
These errors were computed in the same way as in Sec.~\ref{sec:gaussian-mixture} (e.g., the estimated parameters were sorted according to the magnitudes of the mixing coefficients).
Results show that our method performed the best, confirming its effectiveness.
In the Supplementary Materials, we also report the point estimates made using the proposed method, showing that the true parameters were estimated reasonably accurately.
\begin{table}[t]
  \caption{Results for the pedestrian simulator in Sec.~\ref{sec:pedestrian-sim}} \label{tab:table3}
  \begin{center}
  \begin{adjustbox}{max width=220pt}
  \begin{tabular}{|l|c|r|r|r|r|} \hline
    Algorithm & $\theta^{(N)}$ error & $\theta^{(T)}$ error & data error& cputime \\ \hline
    KR-ABC & 61.58(74.42) & 70.93(102.08) & 0.008(0.009) & 2233.45(97.54)\\ \hline
    KR-ABC (less) & 82.46(75.05) & 134.00(161.85) & 0.014(0.014) & 1875.32(147.16)\\ \hline
    K2-ABC & 298.94(120.71) & 308.95(109.43) & 0.10(0.10)& 1547.32(56.31)\\ \hline
    K-ABC & 354.72(145.76) & 389.52(140.91) & 0.12(0.09)& 1773.74(84.91)\\ \hline
    SMC-ABC (mean) & 271.51(104.64) & 363.12(91.28) & 0.09(0.07)& 2017.89(110.02)\\ \hline
    SMC-ABC (MAP) & 255.15(139.33) & 348.43(104.74) & 0.09(0.1)& 2017.89(110.02)\\ \hline
    ABC-DC & 273.93(136.14)& 327.48(98.12) & 0.09(0.14) & 1984.43(59.12)\\ \hline
    BO & 194.57(65.83) & 291.73(105.33) & 0.04(0.06) & 37541.23(3047.46)\\ \hline
    MSM & 453.58(89.43) & 510.04(55.10) & 0.24(0.17) & 1869.83(49.51)\\ \hline

  \end{tabular}
  \end{adjustbox}
  \end{center}
  \vspace{-0.25in}
\end{table}

\section{Summary and future work} \label{sec:conclusions}
We have proposed kernel recursive ABC for point estimation with intractable likelihood and have empirically investigated the effectiveness of this approach. While we have also provided theoretical analysis to a certain extent, there remain important theoretical topics, as discussed in Sec.~\ref{sec:theory}, that we wish to reserve for future research.

\subsection*{Acknowledgements}
We thank the anonymous reviewers as well as Itaru Nishioka, Itsuki Noda, Takashi Washio, Shinji Ito, Wittawat Jitkrittum, and Marie Oshima for their helpful comments and support. We also thank Shuhei Mano for providing his code.
MK has been supported by the European Research Council (StG Project PANAMA).
KF has been supported by JSPS KAKENHI 26280009.

\bibliographystyle{natbib}

redpajama_set_name: RedPajamaArXiv


Looking for a hotel near I-55 and I-40 in Memphis, TN? Hotels-Rates has you covered with everything you want to know about lodging and hotels around this exit and elsewhere in Memphis. Whether you are traveling for work or pleasure, Hotels-Rates has information on all the things to do, attractions and accomodations in Memphis. Headed back to school? Victory University, Baptist College of Health Sciences, Southern College of Optometry, Memphis College of Art, University of Tennessee - Health Sciences Center, LeMoyne-Owen College, University of Memphis, Christian Brothers University and Rhodes College are relatively close to the exit, with Victory University being the closest. If you are looking for other things to do in Memphis, make sure you check out the Orpheum Theatre, Chickasaw Gardens, Hard Rock Cafe, Memphis Zoo and Graceland, most of which are convenient from the exit. If you are into sports, you will be excited to find that Memphis Grizzlies - FedEx Forum, Memphis Tigers Football - Liberty Bowl Memorial Stadium, Memphis Motorsports Park, AutoZone Liberty Bowl, Memphis Tigers Basketball - FedEx Forum and Pyramid Arena are close to the exit.


Composed by Stephen DeCesare, George Elvey. Christian, Contemporary Christian, Sacred, Praise & Worship, Easter. Sheet Music Single. 16 pages. Published by Exultet Music (S0.2691).
Composed by Stephen DeCesare, George Elvey. Christian, Contemporary Christian, Sacred, Praise & Worship, Easter. Sheet Music Single. 15 pages. Published by Exultet Music (S0.469253).
Find scores for Stephen Elvey at Sheet Music Plus.


MedStar Authors catalog › Details for: Progress for mitral replacement in mitral annular calcification.
Details for: Progress for mitral replacement in mitral annular calcification.
Progress for mitral replacement in mitral annular calcification.
Citation: Journal of Thoracic & Cardiovascular Surgery. 2018 Nov 27.Journal: The Journal of thoracic and cardiovascular surgery.Published: ; 2018ISSN: 0022-5223.Full author list: Edelman JJ; Khan JM; Thourani VH.UI/PMID: 30612753.Subject(s): IN PROCESS -- NOT YET INDEXEDInstitution(s): MedStar Heart & Vascular InstituteActivity type: Journal Article.Medline article type(s): EditorialDigital Object Identifier: https://dx.doi.org/10.1016/j.jtcvs.2018.11.062 (Click here) Abbreviated citation: J Thorac Cardiovasc Surg. 2018 Nov 27.Local Holdings: Available online from MWHC library: 1994 - present, Available in print through MWHC library: 1999 - 2006.


For more information and a quote, check boxes that apply and fill out the form below.
EnerGroup, LLC. is an oil and gas services company located in Houston, Texas. Strategically aligned with manufacturers, fabricators and packagers worldwide. We offer specialized Engineering, Procurement, Consulting, Construction and Financing services geared to the Upstream – Midstream – Downstream sectors. Our company provides leading edge equipment solutions and unparalleled service support to domestic and international onshore/offshore markets from the Wellhead to the Grid.
The exploration & production business is in a state of transformation. With the development of horizontal drilling & fracking technology, a surplus of crude & natural gas was produced from the U.S. shale boom – paired with stagnant global energy demand, it has created a challenging outlook for the industry. Companies are forced to do more with less to continue operating under unabated low oil & gas prices. We help our upstream clients navigate the difficult market environment by creating efficiencies with intuitive 3D engineering & design to the selection of quality products at competitive prices from our team of forward-thinking manufacturers & their products & services. From the wellhead to the sales meter, EnerGroup has your project covered with extraordinary products & services.
The U.S. midstream sector is the backbone of the oil & gas industry. Growth in this sector was spurred by the significant increase in domestic oil & gas production created by the technologies gained through more efficient drilling & completion methods. This increase has played a vital role in moving the surplus of crude & natural gas resources from the wellhead to the downstream customers through construction or expansion of new pipeline, gathering & processing plants across the United States. From component level valves, instruments & meters to complete Production/Processing plants, EnerGroup has a diverse line of products available to support your next project.
Downstream management teams face the ongoing challenges of an aging refining & distribution infrastructure. As refineries are spun-off from vertically integrated oil & gas companies, executives are tasked with building organizations to manage the underlying processes & technology – sometimes without proper documentation & drawings to support the existing operations. EnerGroup's team has the experience & knowledge to equip downstream organizations with the proper documentation & has built 3D drawings with their cutting-edge technology "VREF" (Virtual Replication of Existing Facilities). With our proprietary VREF capability, we are able to recreate component level products to full line processing plants in an environment that is fail-safe, giving HSE departments the opportunity to utilize our teach/train/test mode for field service & operations personnel. From simple plant modifications to full scale recreation of existing facilities EnerGroup has your project covered.
For over 50 years, EnerGroup has lead oilfield service & manufacturing companies through every economic cycle of the volatile oil & gas market, both from the management seat, board member or as trusted advisors. Our team is experienced in all facets of the oilfield service & manufacturing sectors including fabrication, equipment sales & rental, & field service operations. We're here to serve, streamline & support our clients' operations through efficiencies gained in utilizing forward-thinking engineering & design, alignment through more competitive supply chain of products & services to consulting, procurement & project management for job specific projects. As our industry navigates through the challenges in rebuilding our industry & companies, EnerGroup is positioned to support our clients' unexpected project demands by offering outsourced services as needed. No other group can offer the diverse cost-saving opportunities from engineering, to more competitive products through our supply chain of manufacturers to our customer support services.
QUALITY AND PROFESSIONALISM YOU CAN TRUST.
As the Oil and Gas market navigates through these unpredictable times, consider outsourcing with EnerGroup. Our team of talented and experienced professionals are available to help you fill your void when the next unexpected project hits. Why stress your current staff or speculate over the need to hire full-time staff and encumber your business with overhead, vacations, health insurance, 401k etc…only to disappoint and have to endure another round of demoralizing, de-energizing RIFs.
Lean on EnerGroup for your project support needs.
I have known and done business with Kurt Wind since 1991 as a customer for his products, a vendor for Kurt's manufacturing business, and business partner in the upstream oil and gas process equipment business. As a customer for Wind Production related companies, Kurt's knowledge and integrity for quality always produced products a customer can rely on to do the job right the first time. As a vendor for those same companies, I know Kurt to be honorable with his commitments, and fair with his vendors. Finally, as a past business partner, I found Kurt to be a seasoned manager of the highest integrity. Additionally, Kurt possesses the gift of creative foresight to somehow always remain ahead of the pack in whatever venture he undertakes.
EnerGroup has exceeded our expectations. I have always appreciated their commitment to customer service. That's what keeps us coming back. Strategic Energy products can raise our customer service levels, thanks to Kurt and the EnerGroup organization.
Kurt has shown himself to be the consummate professional, expert in his field and through the years, Kurt has shown innovation throughout his career while remaining steadfast in his beliefs that his efforts must benefit his customers and that his team is paramount to the success of the undertaking. Kurt is a man of his word and in many ways is 'old school' and still believes in shaking a person's hand and having a deal. At the same time, he recognizes innovation in technology, manufacturing and leadership while utilizing the best tools at his disposal to solve complex problems at the customer level. While at KWI, a company he helped build with his family, as President, he led it to weather downturns, rapid growth and product changes all while staying true to his customers and employees. In the past, Kurt has been both a competitor and a vendor with organizations I have been associated with and in those interactions he has presented himself with the utmost professionalism and fairness in his dealings with us. His newest venture with EnerGroup shows his latest innovation in taking the rep or supply chain partner concept into the fabricated and custom built product markets. If history is a predictor of future results, this venture will be successful and beneficial to the end users that he serves.
My name is Danny Ford and I have been involved in the Oil & Gas Industry for over 30 years. I have the honor of writing a Letter of Recommendation for a very good friend of mine, Kurt Wind. I have known Kurt for about 10 years but have known of him for many more as he and his dad, Ken Wind, are legends in the industry. Kurt had taken a very respected family owned and operated company and turned it into a world class operation recognized around the globe and continues that record with his most recent venture EnerGroup. Kurt's knowledge of the industry, outstanding integrity and warm personality has allowed him to excel at anything he has attempted and along with foresight and innovation, I fully expect him to accomplish anything he sets his mind to. I highly value Kurt as a personal friend and I have enjoyed the years since we first met. On a professional level, he has been an inspiration to me as well as anyone he comes into contact with. His future will always be bright and I will always cherish our relationship.