text:
Identification of novel, cryptic Clostridioides species isolates from environmental samples collected from diverse geographical locations
Charles Hall Davis Williamson, Nathan E. Stone, Amalee E. Nunnally, Chandler C. Roe, Adam J. Vazquez, Samantha A. Lucero, Heidie Hornstra, David M. Wagner, Paul Keim, Maja Rupnik, Sandra Janezic, Jason William Sahl
Clostridioides difficile is a pathogen often associated with hospital-acquired infection or antimicrobial-induced disease; however, increasing evidence indicates infections can result from community or environmental sources. Most genomic sequencing of C. difficile has focused on clinical strains, although evidence is growing that C. difficile spores are widespread in soil and water in the environment. In this study, we sequenced 38 genomes collected from soil and water isolates in Flagstaff (AZ, USA) and Slovenia in an effort targeted towards environmental surveillance of C. difficile. At the average nucleotide identity (ANI) level, the genomes were divergent to C. difficile at a threshold consistent with different species. A phylogenetic analysis of these divergent genomes together with Clostridioides genomes available in public repositories confirmed the presence of three previously described, cryptic Clostridioides species and added two additional clades. One of the cryptic species (C-III) was almost entirely composed of Arizona and Slovenia genomes, and contained distinct sub-groups from each region (evidenced by SNP and gene-content differences). A comparative genomics analysis identified multiple unique coding sequences per clade, which can serve as markers for subsequent environmental surveys of these cryptic species. Homologues to the C. difficile toxin genes, tcdA and tcdB, were found in cryptic species genomes, although they were not part of the typical pathogenicity locus observed in C. difficile, and in silico PCR suggested that some would not amplify with widely used PCR diagnostic tests. We also identified gene homologues in the binary toxin cluster, including some present on phage and, for what is believed to be the first time, on a plasmid. All isolates were obtained from environmental samples, so the function and disease potential of these toxin homo-logues is currently unknown. Enzymatic profiles of a subset of cryptic isolates (n=5) demonstrated differences, suggesting that these isolates contain substantial metabolic diversity. Antimicrobial resistance (AMR) was observed across a subset of isolates (n=4), suggesting that AMR mechanisms are intrinsic to the genus, perhaps originating from a shared environmental origin. This study greatly expands our understanding of the genomic diversity of Clostridioides. These results have implications for C. difficile One Health research, for more sensitive C. difficile diagnostics, as well as for understanding the evolutionary history of C. difficile and the development of pathogenesis.
https://doi.org/10.1099/mgen.0.000742
Clostridioides difficile
cryptic species
10.1099/mgen.0.000742
Dive into the research topics of 'Identification of novel, cryptic Clostridioides species isolates from environmental samples collected from diverse geographical locations'. Together they form a unique fingerprint.
Genome Medicine & Life Sciences 100%
Slovenia Medicine & Life Sciences 68%
One Health Medicine & Life Sciences 37%
Environmental Monitoring Medicine & Life Sciences 35%
Water Medicine & Life Sciences 32%
Polymerase Chain Reaction Medicine & Life Sciences 29%
Williamson, C. H. D., Stone, N. E., Nunnally, A. E., Roe, C. C., Vazquez, A. J., Lucero, S. A., Hornstra, H., Wagner, D. M., Keim, P., Rupnik, M., Janezic, S., & Sahl, J. W. (2022). Identification of novel, cryptic Clostridioides species isolates from environmental samples collected from diverse geographical locations. Microbial Genomics, 8(2), [000742]. https://doi.org/10.1099/mgen.0.000742
Identification of novel, cryptic Clostridioides species isolates from environmental samples collected from diverse geographical locations. / Williamson, Charles Hall Davis; Stone, Nathan E.; Nunnally, Amalee E. et al.
In: Microbial Genomics, Vol. 8, No. 2, 000742, 2022.
Williamson, CHD, Stone, NE, Nunnally, AE, Roe, CC, Vazquez, AJ, Lucero, SA, Hornstra, H, Wagner, DM, Keim, P, Rupnik, M, Janezic, S & Sahl, JW 2022, 'Identification of novel, cryptic Clostridioides species isolates from environmental samples collected from diverse geographical locations', Microbial Genomics, vol. 8, no. 2, 000742. https://doi.org/10.1099/mgen.0.000742
Williamson CHD, Stone NE, Nunnally AE, Roe CC, Vazquez AJ, Lucero SA et al. Identification of novel, cryptic Clostridioides species isolates from environmental samples collected from diverse geographical locations. Microbial Genomics. 2022;8(2):000742. doi: 10.1099/mgen.0.000742
Williamson, Charles Hall Davis ; Stone, Nathan E. ; Nunnally, Amalee E. et al. / Identification of novel, cryptic Clostridioides species isolates from environmental samples collected from diverse geographical locations. In: Microbial Genomics. 2022 ; Vol. 8, No. 2.
@article{99f6750599474d6db3f52c527069b204,
title = "Identification of novel, cryptic Clostridioides species isolates from environmental samples collected from diverse geographical locations",
abstract = "Clostridioides difficile is a pathogen often associated with hospital-acquired infection or antimicrobial-induced disease; however, increasing evidence indicates infections can result from community or environmental sources. Most genomic sequencing of C. difficile has focused on clinical strains, although evidence is growing that C. difficile spores are widespread in soil and water in the environment. In this study, we sequenced 38 genomes collected from soil and water isolates in Flagstaff (AZ, USA) and Slovenia in an effort targeted towards environmental surveillance of C. difficile. At the average nucleotide identity (ANI) level, the genomes were divergent to C. difficile at a threshold consistent with different species. A phylogenetic analysis of these divergent genomes together with Clostridioides genomes available in public repositories confirmed the presence of three previously described, cryptic Clostridioides species and added two additional clades. One of the cryptic species (C-III) was almost entirely composed of Arizona and Slovenia genomes, and contained distinct sub-groups from each region (evidenced by SNP and gene-content differences). A comparative genomics analysis identified multiple unique coding sequences per clade, which can serve as markers for subsequent environmental surveys of these cryptic species. Homologues to the C. difficile toxin genes, tcdA and tcdB, were found in cryptic species genomes, although they were not part of the typical pathogenicity locus observed in C. difficile, and in silico PCR suggested that some would not amplify with widely used PCR diagnostic tests. We also identified gene homologues in the binary toxin cluster, including some present on phage and, for what is believed to be the first time, on a plasmid. All isolates were obtained from environmental samples, so the function and disease potential of these toxin homo-logues is currently unknown. Enzymatic profiles of a subset of cryptic isolates (n=5) demonstrated differences, suggesting that these isolates contain substantial metabolic diversity. Antimicrobial resistance (AMR) was observed across a subset of isolates (n=4), suggesting that AMR mechanisms are intrinsic to the genus, perhaps originating from a shared environmental origin. This study greatly expands our understanding of the genomic diversity of Clostridioides. These results have implications for C. difficile One Health research, for more sensitive C. difficile diagnostics, as well as for understanding the evolutionary history of C. difficile and the development of pathogenesis.",
keywords = "Clostridioides difficile, cryptic species, genomics, toxin",
author = "Williamson, {Charles Hall Davis} and Stone, {Nathan E.} and Nunnally, {Amalee E.} and Roe, {Chandler C.} and Vazquez, {Adam J.} and Lucero, {Samantha A.} and Heidie Hornstra and Wagner, {David M.} and Paul Keim and Maja Rupnik and Sandra Janezic and Sahl, {Jason William}",
note = "Funding Information: This research was funded in part by the Flinn Foundation (http://www.flinn.org/), grants Flinn 2234 and Flinn 2062. The funders had no role in study design, data collection and analysis, decision to publish nor preparation of the manuscript Publisher Copyright: {\textcopyright} 2022 The Authors.",
doi = "10.1099/mgen.0.000742",
journal = "Microbial genomics",
publisher = "Microbiology Society",
T1 - Identification of novel, cryptic Clostridioides species isolates from environmental samples collected from diverse geographical locations
AU - Williamson, Charles Hall Davis
AU - Stone, Nathan E.
AU - Nunnally, Amalee E.
AU - Roe, Chandler C.
AU - Vazquez, Adam J.
AU - Lucero, Samantha A.
AU - Hornstra, Heidie
AU - Wagner, David M.
AU - Keim, Paul
AU - Rupnik, Maja
AU - Janezic, Sandra
AU - Sahl, Jason William
N1 - Funding Information: This research was funded in part by the Flinn Foundation (http://www.flinn.org/), grants Flinn 2234 and Flinn 2062. The funders had no role in study design, data collection and analysis, decision to publish nor preparation of the manuscript Publisher Copyright: © 2022 The Authors.
N2 - Clostridioides difficile is a pathogen often associated with hospital-acquired infection or antimicrobial-induced disease; however, increasing evidence indicates infections can result from community or environmental sources. Most genomic sequencing of C. difficile has focused on clinical strains, although evidence is growing that C. difficile spores are widespread in soil and water in the environment. In this study, we sequenced 38 genomes collected from soil and water isolates in Flagstaff (AZ, USA) and Slovenia in an effort targeted towards environmental surveillance of C. difficile. At the average nucleotide identity (ANI) level, the genomes were divergent to C. difficile at a threshold consistent with different species. A phylogenetic analysis of these divergent genomes together with Clostridioides genomes available in public repositories confirmed the presence of three previously described, cryptic Clostridioides species and added two additional clades. One of the cryptic species (C-III) was almost entirely composed of Arizona and Slovenia genomes, and contained distinct sub-groups from each region (evidenced by SNP and gene-content differences). A comparative genomics analysis identified multiple unique coding sequences per clade, which can serve as markers for subsequent environmental surveys of these cryptic species. Homologues to the C. difficile toxin genes, tcdA and tcdB, were found in cryptic species genomes, although they were not part of the typical pathogenicity locus observed in C. difficile, and in silico PCR suggested that some would not amplify with widely used PCR diagnostic tests. We also identified gene homologues in the binary toxin cluster, including some present on phage and, for what is believed to be the first time, on a plasmid. All isolates were obtained from environmental samples, so the function and disease potential of these toxin homo-logues is currently unknown. Enzymatic profiles of a subset of cryptic isolates (n=5) demonstrated differences, suggesting that these isolates contain substantial metabolic diversity. Antimicrobial resistance (AMR) was observed across a subset of isolates (n=4), suggesting that AMR mechanisms are intrinsic to the genus, perhaps originating from a shared environmental origin. This study greatly expands our understanding of the genomic diversity of Clostridioides. These results have implications for C. difficile One Health research, for more sensitive C. difficile diagnostics, as well as for understanding the evolutionary history of C. difficile and the development of pathogenesis.
AB - Clostridioides difficile is a pathogen often associated with hospital-acquired infection or antimicrobial-induced disease; however, increasing evidence indicates infections can result from community or environmental sources. Most genomic sequencing of C. difficile has focused on clinical strains, although evidence is growing that C. difficile spores are widespread in soil and water in the environment. In this study, we sequenced 38 genomes collected from soil and water isolates in Flagstaff (AZ, USA) and Slovenia in an effort targeted towards environmental surveillance of C. difficile. At the average nucleotide identity (ANI) level, the genomes were divergent to C. difficile at a threshold consistent with different species. A phylogenetic analysis of these divergent genomes together with Clostridioides genomes available in public repositories confirmed the presence of three previously described, cryptic Clostridioides species and added two additional clades. One of the cryptic species (C-III) was almost entirely composed of Arizona and Slovenia genomes, and contained distinct sub-groups from each region (evidenced by SNP and gene-content differences). A comparative genomics analysis identified multiple unique coding sequences per clade, which can serve as markers for subsequent environmental surveys of these cryptic species. Homologues to the C. difficile toxin genes, tcdA and tcdB, were found in cryptic species genomes, although they were not part of the typical pathogenicity locus observed in C. difficile, and in silico PCR suggested that some would not amplify with widely used PCR diagnostic tests. We also identified gene homologues in the binary toxin cluster, including some present on phage and, for what is believed to be the first time, on a plasmid. All isolates were obtained from environmental samples, so the function and disease potential of these toxin homo-logues is currently unknown. Enzymatic profiles of a subset of cryptic isolates (n=5) demonstrated differences, suggesting that these isolates contain substantial metabolic diversity. Antimicrobial resistance (AMR) was observed across a subset of isolates (n=4), suggesting that AMR mechanisms are intrinsic to the genus, perhaps originating from a shared environmental origin. This study greatly expands our understanding of the genomic diversity of Clostridioides. These results have implications for C. difficile One Health research, for more sensitive C. difficile diagnostics, as well as for understanding the evolutionary history of C. difficile and the development of pathogenesis.
KW - Clostridioides difficile
KW - cryptic species
KW - genomics
KW - toxin
U2 - 10.1099/mgen.0.000742
DO - 10.1099/mgen.0.000742
JO - Microbial genomics
JF - Microbial genomics


text:
Ramsey rewarded with new contract
by Randy O. on June 1, 2010 Ramsey rewarded with new contract2010-06-01T07:54:35-05:00 - Arsenal
Aaron Ramsey has signed a new contract at Arsenal.
The 19-year-old has impressed both on the international stage for Wales and for Arsenal ever since the club recruited him from Cardiff City in the summer of 2008.
Ramsey's influential shows in midfield has seen him tipped as a future replacement for the unsettled Cesc Fabregas but the Welshman faced a potential dim to his bright future in the severe leg-break he suffered at the hands of Ryan Shawcross earlier this year.
Arsenal's No. 16 has been sidelined since late February and is unlikely to return anytime soon, but the courage he has shown in his recovery process has earned him a new deal at the club.
Thus, Ramsey has expressed his delight at signing a new deal.
"I'm very happy here at Arsenal, it's a great club and I've learned a lot since I've been here"
"Every player wants to play in a team where they can play football and I feel that this is the way it should be played, so that's one of the reasons I signed a new contract."
"I think I can still develop my game a lot, and I'm in the right place to do that with the support of the manager and also a squad of very talented players. I'm convinced the next few years will be a time where my career kicks on to a new level, it's an exciting time to be part of the Club."
6 Comments Already
Matlee - June 1st, 2010 at 9:56 AM
so what does that mean? Cesc has recently signed a long contract and the fans still don't know if he is staying or going. As far as i can see signing a new contract is just improved terms for the players, but they can still leave whenever they want. So why bother to make big announcements about contracts, because it certainly does not assure us fans that the player will remain at the club for the full duration of that contract.
Randy Osae - June 1st, 2010 at 10:02 AM
You are spot on man. Contracts only mean improved wages for the player. It has nothing to do with the future or loyalty of the player being secured or anything.
Henry signed a new five-year contract in 2006 and then joined Barca a year later. Some clubs even do it to just to put a price on the player (especially if he is close to being a free agent) to make a profit on selling him.
New contracts have more to do with money than loyalty.
But you may ask, why I'm I reporting this then? Because Arsenal FC said it's news!
arsenekidz - June 1st, 2010 at 11:29 AM
Good for you, Ramsey.
Loft - June 1st, 2010 at 11:43 AM
I just hope he comes back stronger and not end up like, Eduardo did when he came back. Sad really. I wonder how, Eduardo will respond next season.
When is, Ramsey able to train again? Will he be ready for pre-season?
yakubu m.odah - June 1st, 2010 at 3:29 PM
Is a good one 4 him i wish him all the best.
Matlee - June 2nd, 2010 at 5:35 AM
to Randy: not blaming you for article
i understand you are just reporting the news.
Question: Barcelona protect themselves by placing 200 mil buy out clauses on their top players such as Messi, David Villa. So it is very unlikely to hear rumors about a club coming in for them.
Why didn't Arsenal think of doing something similar with their top players? then unless the buy out fee is settled players have to complete their contract. even if the clause is reached it is so much money its a win win for the club.
« Walcott excluded from England squad
VIDEO: Van Persie involved in two goals again for Dutch »

redpajama_set_name: RedPajamaCommonCrawl


A Potain MD 689 M40 top-slewing crane is helping to construct the new 'BioDome' in the famous ZooParc de Beauval in Saint-Augnan-sur-Cher, France.
The MD 689 M40 was installed in march 2018 and is now nearing the end of the project. Dismantling is planned for February 2019. When complete, the BioDome will accommodate 3,000 visitors and house Komodo dragons, hippos and manatees.


text:
\subsubsection*{References}}
\usepackage{mathtools}
\usepackage{booktabs}
\usepackage{tikz}

\newcommand{\swap}[3][-]{#3#1#2}

\newcommand{\myEqnDeco}[1]{
\mydeco{subequations}{
\mydeco{eqnarray}{
#1}}
}

\input{front}

\input{myPKG}

\title{Residual Bootstrap Exploration for Stochastic Linear Bandit}

\author[1]{Shuang Wu}
\author[1,2]{Chi-Hua Wang}
\author[1]{Yuantong Li}
\author[1]{Guang Cheng}
\affil[1]{%
    Department of Statistics\\
    University of California, Los Angeles\\
    Los Angeles, California, USA
}
\affil[2]{%
    Department of Statistics\\
    Purdue University\\
    West Lafayette, Indiana, USA
}

\begin{document}
\maketitle

\begin{abstract}
We propose a new bootstrap-based online algorithm for stochastic linear bandit problems. The key idea is to adopt residual bootstrap exploration, in which the agent estimates the next step reward by re-sampling the residuals of mean reward estimate. Our algorithm, residual bootstrap exploration for stochastic linear bandit (\texttt{LinReBoot}), estimates the linear reward from its re-sampling distribution and pulls the arm with the highest reward estimate. In particular, we contribute a theoretical framework to demystify residual bootstrap-based exploration mechanisms in stochastic linear bandit problems. The key insight is that the strength of bootstrap exploration is based on collaborated optimism between the online-learned model and the re-sampling distribution of residuals. Such observation enables us to show that the proposed \texttt{LinReBoot} secure a high-probability $\Tilde{O}(d \sqrt{n})$ sub-linear regret under mild conditions. Our experiments support the easy generalizability of the \texttt{ReBoot} principle in the various formulations of linear bandit problems and show the significant computational efficiency of \texttt{LinReBoot}.
\end{abstract}

\section{Introduction}\label{sec:intro}

Stochastic linear bandit is an online learning problem that the learning agent acts by pulling arms, where each arm is associated with a feature vector, then learning the arms information from the corresponding random rewards. In such problems, the typical goal of a learning agent is to maximize its cumulative reward.
Learning more about an arm (explore) or pulling the arm with the highest estimated reward (exploit) leads to the well-known \textit{exploration- exploitation trade-off}, which is the central trade-off captured in many decision-making applications in modern online service industries.
Consequently, the design of stochastic linear bandit algorithms demands an easy-generalizable implementation across various contextualize actions and reward generation processes.

In the past decade of bandit literature, such demands have invited researchers to investigate bootstrap-based exploration-exploitation trade-offs and have drawn rising attention \citep{baransi2014sub, eckles2014thompson, osband2015bootstrapped, vaswani2018new, hao2019bootstrapping, kveton2019garbage, wang2020residual}. Yet, prior works on bootstrap-based bandit algorithms focus on provable multi-armed bandit algorithms and only provide a limited empirical evaluation of bootstrap-based stochastic linear bandit algorithms, and their theoretical counterpart remains unknown. Such knowledge gap of bootstrapping stochastic linear bandit persuades our investigation on the provable bootstrap-based stochastic linear bandits: \textbf{Can we theoretically and empirically support the validity and easy-generalizability of bootstrapping procedure in stochastic linear bandit algorithms design?} In particular, we aim to deliver a generic framework to demystify the bootstrap optimism in stochastic linear bandit problems and validate the easy generalizability of the bootstrap principle across various contextual linear bandit problems.

\textbf{Contributions.}
We introduce \texttt{LinReBoot} algorithms that implement Residual Bootstrap Exploration for stochastic linear bandit problem with sub-linear regret. We theoretically show that \texttt{LinReBoot} secures $\Tilde{O}(d \sqrt{n})$ regret where $d$ is the dimension of features. This sub-linear regret bound matches the regret bound of the same order as those theoretical results of Linear Thompson Sampling algorithms. The key to achieving such sub-linear regret guarantee is to carefully manage and collaborate sample and bootstrap optimism (Section \ref{sec:colla_opti}). In particular, by measuring the ''sample-bootstrap optimistic estimated discrepancy ratio'' of the optimal arm, \texttt{LinReboot} successfully avoids over or under exploration and theoretically secures sub-linear mean regret with high-probability. To our knowledge, this is the first theoretical analysis to support the validity and efficiency of the residual bootstrap-based procedure for stochastic linear bandit problems. We empirically show that \texttt{LinReBoot} rivals or exceeds competing algorithms including Linear Thompson Sampling, Linear PHE, Linear GIRO, and Linear UCB under stochastic linear bandit problem as well as more complicated linear bandit settings. These significant results support the easy-generalizability of proposed \texttt{LinReBoot}. In summary, our contributions are as follows:
\begin{itemize}[leftmargin=5pt, itemsep = -2pt]
\item Propose \texttt{LinReBoot} algorithms that implement Residual Bootstrap Exploration in linear bandit problems without boundness assumption of rewards.
\item  Theoretically show that \texttt{LinReBoot} secures $\Tilde{O}(d \sqrt{n})$ regret, matching the regret bound of the same order as those theoretical results of Linear Thompson Sampling algorithms.
\item Empirically show that \texttt{LinReBoot} rivals or exceeds baseline algorithms and supports that \texttt{LinReBoot} is easy-generalizable among linear bandit problems.
\end{itemize}

\textbf{Related Works.}
Bootstrap-based contextual bandit algorithms design has been actively studied in the last half-decade and drawn a surge of interest from both theoretical studies and industrial practice \citep{elmachtoub2017practical, eckles2014thompson, osband2016deep, kveton2019garbage, hao2019bootstrapping}. Bootstrap-based bandit algorithm design is a paradigm of sequential decision-making based on an exploration mechanism with no pre-defined mean reward model. Such paradigm enjoys a decisive advantage that engineers are free to deploy any reward model of interests without painful adaption to problem structure \citep{kveton2019garbage, kveton2019perturbedMAB}. \texttt{ReBoot} \citep{wang2020residual} provided a theoretical logarithmic regret guarantee for multi-armed bandit (MAB) and empirical investigation to validate the easy generalizability of the \texttt{ReBoot} principle. Our work aims to provide a theoretical guarantee for the bootstrap-based linear bandit algorithms and empirically investigate more general contextual linear bandit setting to validate the \texttt{ReBoot} principle.

One close related work is \citep{kveton2019perturbed} which introduces perturbation of past samples for exploration under stochastic linear bandit problem. The limitation of \citep{kveton2019perturbed} is the boundness of rewards, indicating many broader classes of rewards such as Gaussian rewards are not applicable with a theoretical guarantee. In contrast, the proposed \texttt{LinReBoot} algorithms relax the boundness reward assumption and thus validate bootstrap-based bandit algorithms in wider bandit environments with a broader class of reward generation processes.

Early works about exploration in bandit problems \citep{abbasi2011improved, langford2007epoch, dani2008stochastic} are practical but no guarantee of the optimality. Some works \citep{wang2020residual, kveton2019garbage, kveton2019perturbedMAB, thompson1933likelihood, auer2002finite} provide well designed exploration for bandit problems and have their own principles for adopting to more general problems. In these works, three principles including \texttt{ReBoot}\citep{wang2020residual}, \texttt{GIRO}\citep{kveton2019garbage} and \texttt{PHE}\citep{kveton2019perturbedMAB} are devising exploration mechanism based on up-to-now history instead of on pre-defined reward model in the other two principles \texttt{TS}\citep{thompson1933likelihood} and  \texttt{UCB}\citep{auer2002finite}. Our work generalizes \texttt{ReBoot} into stochastic linear bandit problems.

\textbf{Notations.}
Let $[n]$ be set $\{1,2,...,n\}$. ${\boldsymbol 1}$ is a vector with all ones and ${\boldsymbol I}$ is the identity matrix. For a vector ${\boldsymbol v}$, $\norm{{\boldsymbol v}}_{2}$ is $2$-norm of ${\boldsymbol v}$ and $\norm{{\boldsymbol v}}_{{\boldsymbol A}}^2:=\sqrt{ {\boldsymbol v}^{\top} {\boldsymbol A} {\boldsymbol v}}$ for a semidefinite matrix ${\boldsymbol A}$. Let $\langle \cdot, \cdot \rangle$ be the inner product operation. Denote ${\cal F}_{t}$ as the history of randomness up to round $t$. $\mathbb{E}_{t}[\cdot]:= \mathbb{E}[\cdot|{\cal F}_{t-1}]$ is defined as the conditional expectation given ${\cal F}_{t-1}$ and $\mathbb{P}_{t}(\cdot):= \mathbb{P}(\cdot|{\cal F}_{t-1})$ is defined as the conditional probability given ${\cal F}_{t-1}$. $\mathbb{I}\{\cdot\}$ is indicator function. For a set or event $E$, we denote its complement as $\Bar{E}$. $N(\mu, \sigma^2)$ is Gaussian distribution with mean $\mu$ and variance $\sigma^2$.   We use $\Tilde{O}$ for big $O$ notation up to logarithmic factor.

\section{Stochastic Linear Bandit} \label{sec: Stochastic_Linear_Bandit}

\textbf{Contextualize Action Set.} In stochastic linear bandit problem, we identify the actions with $d-$dimensional features from ${\cal A} \subset \mathbb{R}^{d}$ and assume $|{\cal A}|$, the size of the action set, is finite. Let $K:=|{\cal A}|$ be the number of actions (arms), ${\boldsymbol x}_{k} \in \mathbb{R}^{d}$ be the context vector of the $k$-th arm, that is, ${\cal A} = \{{\boldsymbol x}_1,...,{\boldsymbol x}_K\}$.

\textbf{Reward generating mechanism.} The reward function is parameterized by ${\boldsymbol \theta} \in \mathbb{R}^{d}$ such that, at time $t$ the agent chooses an action $I_{t} \in [K]$ with feature $X_{t} = {\boldsymbol x}_{I_t} \in {\cal A}$, the reward is generated by
\begin{equation}\label{eq:reward_model}
Y_{t} \equiv \langle X_{t}, {\boldsymbol \theta} \rangle + \epsilon_{t} .
\end{equation}
Specifically, the reward obtained by the agent at round $t$ when pulling arm $I_t = k$ is generated from a distribution with mean $\mu_{k}:={\boldsymbol x}_{k}^{\top} {\boldsymbol \theta}$, conditioning on context ${\boldsymbol x}_{k}$. The property of noise $\epsilon_{t}$ is described in Assumption \ref{ass:noise_bound}. Furthermore,  denote the recieved reward by $r_{I_{t}}$ and the reward random variable by $Y_{t}$ at round $t$.

\textbf{Regret.} Without loss of generality, assume that arm $1$ is the unique optimal arm, that is $\mu_1 > \mu_k \mbox{ } \forall k \neq 1$. The optimal gap of the $k$-th arm is $\Delta_{k}: = \mu_1 - \mu_k \geq 0$. The expected $n$-round regret is denoted as
\begin{equation}\label{eq: expected_regret}
R_{n} := \sum_{k=2}^{K} \Delta_{k} \mathbb{E}[\sum_{t=1}^{n} \mathbb{I}\{I_{t} = k\}] .
\end{equation}
The goal of the agent is to maximize the expected cumulative reward in $n$ rounds, which is equivalent to minimizing the expected regret $R_{n}$.

\begin{assumption}\label{ass:bound}(Boundness assumptions)
True parameter ${\boldsymbol \theta}$ is bounded: $\norm{{\boldsymbol \theta}}_2 \leq S_2$.
\end{assumption}
Besides, we denote $L$ as the upper bound for context vectors: $\norm{{\boldsymbol x}_{k}}_2 \leq L$ for all $k \in [K]$.
Assumption \ref{ass:bound} is referred to the boundness assumptions in the stochastic linear bandit literature and is to ensure the regret is bounded if the agent pulls any sub-optimal actions (see Section 5 in \citep{abbasi2011improved}).
\begin{assumption}(Noise Clipping assumption)\label{ass:noise_bound} Noise process $\{\epsilon_{t}\}_{t=1}^{\infty}$ described in \eqref{eq:reward_model} satisfies that for some $L_{1}, L_{2} > 0$,
\begin{equation}
\begin{aligned}
    e^{L_{1} \eta^2} \leq \mathbb{E}[e^{\eta \epsilon_{t}} | {\cal F}_{t-1}] \leq e^{L_{2} \eta^2} \mbox{, } \forall \eta \geq 0,
\end{aligned}
\end{equation}
where ${\cal F}_{t-1}=\{\epsilon_1, I_{1}, \cdots, \epsilon_{t-1}, I_{t-1}\}$.
\end{assumption}
Assumption \ref{ass:noise_bound} implies that stochastic process $\{\epsilon_{t}\}_{t=1}^{\infty}$ is conditionally sub-gaussian with constant $L_{2}$. $L_{1}$ contributes to the lower bound of moment generating function suggested by \citep{zhang2020non}. Note that the Assumption \ref{ass:noise_bound} allows heteroscedasticity among different arms by choosing $L_{2}$ as the largest variance among arms. Such heteroscedasticity consideration arises and has been identified as a challenge in applications of Bayesian optimization \citep{kirschner2021information, cowen2020empirical}.

\section{Residual Bootstrap Exploration}
\label{sec: LinReBoot_alg}

\subsection{\texttt{ReBoot} Principle}
\label{subsec: ReBoot_principle}
This section presents essential proof of concepts to implement \texttt{ReBoot} principle \citep{wang2020residual}. In general, each round of interaction, the decision policy admits four subroutines to implement \texttt{ReBoot} principle: 1) Learning, 2) Fitting, 3) Bootstrapping, and 4) Exploring. Following elaborates on each subroutine:

\textbf{1) Model Learning.} The first subroutine outputs a learned model based on current collected data. Our implementation learns the parameter ${\boldsymbol \theta}$ in Eq.\eqref{eq:reward_model} by some user-specified model.

\textbf{2) Data Fitting.} The second subroutine fits the current data set with the learned model in the previous subroutine and then outputs the residual set.
Intuitively, the residuals measure the \textit{goodness of fit} of the learned model and should drop a hint on the right amount of exploration. In other words, the residuals should suggest a right magnitude of exploration bonus in decision policy \eqref{eq:policy}. How to manage and integrate uncertainty behind residuals into the exploration mechanism of policy is the main challenge.

\textbf{\textbf{3)} Residuals Bootstraping.} The third subroutine associates the residuals obtained the last subroutine with a bootstrapping distribution. Instead of maintaining a belief distribution on a parameter in the Bayesian approach, \texttt{ReBoot} principle maintains a bootstrapping distribution on the statistical error based on residuals.
The challenge is to justify the efficacy of residual-based optimism construction in both theory and practice.

\textbf{4) Actions Exploring.} The fourth subroutines sample the exploration bonus from the bootstrapping distribution and output an index for each action. Such bootstrap procedure is more computationally efficient than prior efforts since this procedure only requires drawing a sample from the bootstrapping distribution. The challenge is to prove that such bootstrap procedure secures sub-linear regret in theory.

\subsection{\texttt{LinReBoot} Algorithm}
\label{subsec: LinReBoot}
We propose the Linear Residual Bootstrap Exploration algorithm (\texttt{LinReBoot}, Algorithm \ref{alg:LinReBoot: Version_1}) for stochastic linear bandit problems.
This section elaborates the four subroutines in Section \ref{subsec: ReBoot_principle} for the proposed \texttt{LinReBoot}.

\textbf{1)}
\texttt{LinReBoot} uses ridge regression procedure, whose learned parameter is $\Hat{{\boldsymbol \theta}}_{t} $ \eqref{eq:fitted_theta} and estimated mean reward for arm $k$ is $\Hat{\mu}_{k,t}$ \eqref{eq:fitted_mean}. Such way to estimate mean reward is easy to manage the confidence \citep{abbasi2011improved}. Thus, we focus on confidence management for the bootstrap-based exploration.

\textbf{Ridge Regression Procedure.} \texttt{LinReBoot} fits linear model at round $t$ as follow,
\begin{subequations}
\label{def: LSE}
\begin{align}
    {\boldsymbol V}_{t}
    &= {\boldsymbol X}_{t-1}^{\top} {\boldsymbol X}_{t-1} + \lambda {\boldsymbol I}, \\
    \Hat{{\boldsymbol \theta}}_{t}
    &= {\boldsymbol V}_{t}^{-1} {\boldsymbol X}_{t-1}^{\top} {\boldsymbol Y}_{t-1}\label{eq:fitted_theta}, \\
    \Hat{\mu}_{k,t}
    &= {\boldsymbol x}_{k}^{\top} \Hat{{\boldsymbol \theta}}_{t} \mbox{, } \forall k \in [K],
\label{eq:fitted_mean}
\end{align}
\end{subequations}
where ${\boldsymbol X}_{t-1} = (X_{1},...,X_{t-1})^{\top} \in \mathbb{R}^{(t-1) \times d}$. The $\tau$-th row of ${\boldsymbol X}_{t-1}$ is the context $X_{\tau}^{\top}$ for $ \tau \in [t-1]$, ${\boldsymbol Y}_{t-1} = (Y_{1},...,Y_{t-1})^{\top}$ is reward vector whose elements are rewards up to round $t-1$. $\lambda$ denotes the regularization level. ${\boldsymbol V}_{t}$ denotes the sample covariance matrix up to round $t$ and $\Hat{{\boldsymbol \theta}}_{t}$ is the ridge estimation of target parameter ${\boldsymbol \theta}$ in \eqref{eq:reward_model}. $\Hat{\mu}_{k,t}$ denotes the estimated mean of arm $k$ based on history. Note that the first $K$ rounds in proposed \texttt{LinReBoot} is fully exploring each arm once. In other words, $I_t = t$ when $t \in [K]$, indicating ${\boldsymbol X}_{K}:=({\boldsymbol x}_1,...,{\boldsymbol x}_K)^{\top} \in \mathbb{R}^{K \times d}$. We call this ${\boldsymbol X}_{K}$ the context matrix with rank $r \leq \min(K,d)$ and singular values $\sigma_{1}, ... , \sigma_{r}$. Also define $\sigma_{\min}^2 \leq \sigma_{i}^2 \leq \sigma_{\max}^2 \mbox{, } \forall i \in [r]$. With these definitions, we make a mild assumption about the shrinkage effect of ridge regression:

\begin{assumption} (Validity of Ridge Regression)
\label{ass: lower_bound}
The singular value decomposition of context matrix ${\boldsymbol X}_{K}$ is denoted as ${\boldsymbol X}_{K} :={\boldsymbol G} {\boldsymbol \Sigma} {\boldsymbol U}$ where ${\boldsymbol G} \in \mathbb{R}^{K \times K}$, ${\boldsymbol \Sigma} \in \mathbb{R}^{K \times d}$ and ${\boldsymbol U} \in \mathbb{R}^{d \times d}$. Define ${\boldsymbol \Omega} :={\boldsymbol \Sigma} ({\boldsymbol \Sigma}^{\top} {\boldsymbol \Sigma} + \lambda {\boldsymbol I})^{-1} {\boldsymbol \Sigma}^{\top} \in \mathbb{R}^{K \times K}$ and ${\boldsymbol Z} := {\boldsymbol G} {\boldsymbol \Omega} {\boldsymbol \Sigma} {\boldsymbol U} \in \mathbb{R}^{K \times d}$. Let ${\boldsymbol z}_1 \in \mathbb{R}^{d}$ be the first row of ${\boldsymbol Z}$. Given any $\lambda > 0$, there exists a corresponding  positive scalar $S_1$ such that $|{\boldsymbol x}_1^{\top} {\boldsymbol \theta} - {\boldsymbol z}_1^{\top} {\boldsymbol \theta}| \geq S_1$ for the $\theta$ in \eqref{eq:reward_model}.
\end{assumption}

\begin{remark}
Assumption \ref{ass: lower_bound} provides a lower bound of the absolute difference between true mean ${\boldsymbol x}_1^{\top} {\boldsymbol \theta}$ and normalized mean ${\boldsymbol z}_1^{\top} {\boldsymbol \theta}$ of the optimal arm. Note that if $\lambda \rightarrow 0$, then ${\boldsymbol z}_1 \rightarrow {\boldsymbol x}_1$ and $S_1 \rightarrow 0$.
Thus this scalar $S_1$ measures the small perturbation on the mean of the optimal arm when the ridge regression procedure is applied. This ${\boldsymbol Z}$ can be interpreted as a ridge shrinkage context matrix \citep{goldstein1974ridge}. One important phenomenon of online ridge regression is that even if the ridge estimator is biased, the shrinkage effect from ridge estimation provides exploration for the agent leading to making a correct decision. The positive scalar $S_1$ describes the shrinkage effect on the context. That is, the existence of $S_1$ indicates the ridge procedure is valid and its shrinkage effect exists.
\end{remark}

\textbf{2)}
The fitting part of \texttt{LinReBoot} outputs the residuals under the linear model framework,
\begin{equation}
\begin{aligned}
\label{eq: residuals}
    e_{k,t,i}
    &= r_{k,i} - \Hat{\mu}_{k,t} \mbox{, } \forall i \in [s_{k,t-1}],
\end{aligned}
\end{equation}
where $s_{k,t-1}:=\sum_{\tau=1}^{t-1} \mathbb{I}\{I_{\tau} = k\}$ is the number of times pulling arm $k$ by round $t-1$, $r_{k,i}$ is the $i$-th reward of arm $k$ by round $t-1$. The \textit{goodness of fit} of the learned ridge regression model can be summarised by Residual Sum of Squares(RSS) \citep{archdeacon1994correlation} which is defined as
\begin{equation}
\begin{aligned}
    RSS_{k,t} := \sum_{i=1}^{s_{k,t-1}} e_{k,t,i}^2.
\end{aligned}
\end{equation}
Such measure plays an important role in the residual bootstrap exploration mechanism.

\textbf{3)} The third part is Residuals Bootstrapping. This subroutine is independent of the model which suggests the power of generalizability of \texttt{ReBoot} principle. \texttt{ReBoot} principle requires the computation of the exploration bonus \citep{mammen1993bootstrap}, which is $s_{k,t-1}^{-1}\sum_{i=1}^{s_{k,t-1}} \omega_{k,t,i} e_{k,t,i}$, where $\{\omega_{k,t,i}\}_{i=1}^{s_{k,t-1}}$ is residual bootstrap weights for arm $k$ at round $t$.

\begin{algorithm}[t!]
\caption{\texttt{LinReBoot}}\label{alg:LinReBoot: Version_1}
\begin{algorithmic}
\Require $\lambda$, $s_{1,0}=...=s_{K,0}=0$
\For{$t = 1,...,n$}
    \If{$t < K + 1$}
        \State $I_{t} \leftarrow t$
    \Else
        \State ${\boldsymbol V}_{t} \leftarrow {\boldsymbol X}_{t-1}^{\top} {\boldsymbol X}_{t-1} + \lambda {\boldsymbol I}$
        \State $\Hat{{\boldsymbol \theta}}_{t} \leftarrow {\boldsymbol V}_{t}^{-1} {\boldsymbol X}_{t-1}^{\top} {\boldsymbol Y}_{t-1}$
        \For{$k=1,...,K$}
            \State $e_{k,t,i} \leftarrow r_{k,i} - {\boldsymbol x}_{k}^{\top} \Hat{{\boldsymbol \theta}}_{t}$, $\forall i \in \{s_{k,t-1}\}$
            \State Generate $\{\omega_{k,t,i}\}_{i=1}^{s_{k,t-1}}$
            \State $\Tilde{\mu}_{k} \leftarrow {\boldsymbol x}_{k}^{\top} \Hat{{\boldsymbol \theta}}_{t} +
                    s_{k,t-1}^{-1}\sum_{i=1}^{s_{k,t-1}} \omega_{k,t,i} e_{k,t,i} $
        \EndFor
        \State $I_{t} \leftarrow  \underset{k \in [K]}{\arg\max} \mbox{ }  \Tilde{\mu}_{k}$
    \EndIf
    \State $s_{I_{t},t} \leftarrow s_{I_{t},t-1} + 1$ and $s_{k,t} \leftarrow s_{k,t-1}$. $\forall k \neq I_{t}$
    \State Pull arm $I_{t}$ and get reward $r_{I_{t}, s_{I_{t}}}$
    \State
    ${\boldsymbol X}_{t} \leftarrow
    \begin{bmatrix}
    {\boldsymbol X}_{t-1} \\
    {\boldsymbol x}_{I_{t}}^{\top}
    \end{bmatrix}$
    and
    ${\boldsymbol Y}_{t} \leftarrow
    \begin{bmatrix}
    {\boldsymbol Y}_{t-1} \\
    r_{I_{t}, s_{I_{t}}}
    \end{bmatrix}$
\EndFor
\end{algorithmic}
\end{algorithm}

\textbf{Choice of Bootstrapping Weights.} The bootstrap weights considered in this work are i.i.d with zero mean and variance $\sigma_{\omega}^{2}$. They are independent of the noise process $\{\epsilon_{t}\}_{t=1}^{\infty}$. In the literature of bootstrap procedure \citep{mammen1993bootstrap} , the choices of bootstrap weights distribution include Gaussian weights, Rademacher weights and skew correcting weights. In \texttt{LinReBoot}, we adopt the Gaussian bootstrap weights to enable an efficient implement described at section \ref{subsec: Efficient_Implementation}.

\textbf{4)} The last subroutine is the action exploring based on residual bootstrap. More specifically, for arm $k$ at round $t$, \texttt{LinReBoot} adds exploration bonus from residual bootstrapping on the estimated mean $\Hat{\mu}_{k,t}$ as follow,
\begin{equation}
\label{eq:reboot_index}
    \Tilde{\mu}_{k,t}=\Hat{\mu}_{k,t} + \frac{1}{s_{k,t-1}} \sum_{i=1}^{s_{k,t-1}} \omega_{k,t,i} e_{k,t,i},
\end{equation}
then agent pulls arm with the highest bootstrapped mean,
\begin{equation}\label{eq:policy}
    I_{t} \equiv \arg\max_{k \in [K]} \mbox{ } \tilde{\mu}_{k,t}.
\end{equation}
Note that the variance of bootstrapped mean $\Tilde{\mu}_{k,t}$ is $\sigma_{\omega}^2 s_{k,t-1}^{-2}RSS_{k,t}$, indicating an adaptive amount of extra exploration is controlled by $s_{k,t-1}$ and $RSS_{k,t}$.

\textbf{Short Summary.}
Our proposed \texttt{LinReBoot} has following steps at round $t>K$,
\begin{itemize}[leftmargin=15pt, itemsep = -2pt]
    \item[\textbf{1)}] Ridge estimation: compute ${\boldsymbol V}_{t}$, $\Hat{{\boldsymbol \theta}}_{t}$.
    \item[\textbf{2)}] Finding residuals for each arm: for arm $k$, compute $\Hat{\mu}_{k,t}$ and $\{e_{k,t,i}\}_{i=1}^{s_{k,t-1}}$.
    \item[\textbf{3)}] Compute Bootstrapped mean for each arm: for arm $k$, generate $\{\omega_{k,t,i}\}_{i=1}^{s_{k,t-1}}$ and compute $\Tilde{\mu}_{k,t}$ \eqref{eq:reboot_index}.
    \item[\textbf{4)}] Pull arm with the highest $\Tilde{\mu}_{k,t}$ then observe reward.
\end{itemize}

Algorithm \ref{alg:LinReBoot: Version_1} describes \texttt{LinReBoot}. The strength of \texttt{LinReBoot} is its easy generalizability across different bandit problems including linear bandits and even more complicated structured problems (Appendix \ref{appendix: experiment_algs}).

\begin{remark} (\texttt{LinTS} perturbs system parameter estimate, \texttt{LinReBoot} perturbs expected reward estimates)
Compare with the \texttt{LinTS} in \citep{agrawal2013thompson}, in which \texttt{LinTS} samples a perturbed parameter $\tilde{{\boldsymbol \theta}}_{t}^{\texttt{LinTS}}=\hat{{\boldsymbol \theta}}_{t}+\beta_{t} {\boldsymbol V}_{t}^{-1/2} {\boldsymbol \eta}_{t}$ with scaling $\beta_{t}$ and appropriate independent noise ${\boldsymbol \eta}_{t}$ (defined in \citep{agrawal2013thompson}). Our proposed \texttt{LinReBoot} samples a perturbed expected reward $
\tilde{\mu}_{k,t}^{\texttt{LinReBoot}} =\langle \hat{{\boldsymbol \theta}}_{t}, {\boldsymbol x}_{k} \rangle + \frac{1}{s_{k,t-1}}\sum_{i=1}^{s_{k,t-1}}w_{k,t,i}e_{k,t,i}.$ That is, \texttt{LinReBoot} is perturbing the expected reward estimate via prediction error uncertainty, which is supervised by real reward.
In contrast, \texttt{LinTS} is perturbing the system parameter, when can be wrong if the system modeling is wrong.
\end{remark}

\subsection{Efficient Implementation}
\label{subsec: Efficient_Implementation}

By the attractive computational properties of Gaussian distribution, the computational cost of \texttt{LinReBoot} can be reduced significantly when Gaussian Bootstrap weights are generated. Formally: assume $\omega_{k,t,i} \sim N(0, \sigma_{\omega}^{2}) \mbox{, } \forall k, t, i$, recalling \eqref{eq:reboot_index}, for $k\in[K]$ and any $t \geq 1$, bootstrapped mean $\Tilde{\mu}_{k,t}$ follows a Gaussian distribution,
\begin{equation}
\begin{aligned}\label{eq:conditional_distribution_bootstrap_index}
    \Tilde{\mu}_{k,t} | {\cal F}_{t-1} \sim N(\Hat{\mu}_{k,t}, \sigma_{\omega}^2 s_{k,t-1}^{-2}RSS_{k,t}).
\end{aligned}
\end{equation}
Such Gaussian-distributed property of $\Tilde{\mu}_{k,t}$ indicates that if we can update $\Hat{\mu}_{k,t}$, $s_{k,t-1}$ and $RSS_{k,t}$ incrementally for arm $k$, this bootstrapped mean $\Tilde{\mu}_{k,t}$ can be generated by Gaussian generator without inner loop for generating weights. The first two terms, $\Hat{\mu}_{k,t}$ and $s_{k,t-1}$, are naturally updated in incremental manner. For $RSS_{k,t}$, following decomposition ensures an incremental update,
\begin{equation}
\begin{aligned}
\nonumber
    RSS_{k,t}  = \sum_{i=1}^{s_{k,t-1}} r_{k,i}^2 + s_{k,t-1} \Hat{\mu}_{k,t}^2 - 2 \Hat{\mu}_{k,t} \sum_{i=1}^{s_{k,t-1}} r_{k,i}.
\end{aligned}
\end{equation}
Then an efficient generation for $\Tilde{\mu}_{k,t} | {\cal F}_{t-1}$ is ensured by the incremental updates for $\Hat{\mu}_{k,t}$, $s_{k,t-1}$, $\sum_{i=1}^{s_{k,t-1}} r_{k,i}^2$, $\sum_{i=1}^{s_{k,t-1}} r_{k,i}$. Furthermore, since the residual bootstrap weights are generated independently, $\Tilde{\mu}_{k,t}$ among arms are also independent given historical randomness and can be sampled from one multivariate Gaussian generation simultaneously. Formally, $\Tilde{{\boldsymbol \mu}}^{(t)} = (\Tilde{\mu}_{1,t},\dots, \Tilde{\mu}_{K,t})^\top$ is conditional distributed as
\begin{equation}
\begin{aligned}
    \Tilde{{\boldsymbol \mu}}^{(t)} | {\cal F}_{t-1} \sim N_{K}(\Hat{{\boldsymbol \mu}}^{(t)}, {\boldsymbol \Sigma}_{\omega}^{(t)}),
\end{aligned}
\end{equation}
where $\Hat{{\boldsymbol \mu}}^{(t)} = (\Hat{\mu}_{1,t}, \dots, \Hat{\mu}_{K,t})^\top$ and ${\boldsymbol \Sigma}_{\omega}^{(t)}$ is a diagonal matrix with diagonal elements $\sigma_{\omega}^2 s_{k,t-1}^{-2}RSS_{k,t}$. Detailed steps and more illustration about efficient implementation is provided in Appendix \ref{appendix: experiment_efficient}. Moreover, an empirical study about computational efficiency is conducted in Appendix \ref{appendix: experiment_comp_cost} and Table.\ref{table: comp_cost} provides the computational cost of our proposed \texttt{LinReBoot} as well as other baseline algorithms.

\section{Optimism design}
\label{sec:TheoConsi}

\textbf{Optimistic Estimated Discrepancy.}
This section identifies and demystifies the technical challenge of implementing \texttt{ReBoot} principle in the stochastic linear bandit problem. The key is to conduct a detailed investigation to produce probabilistic control on the behavior of  the '\textbf{O}ptimistic \textbf{E}stimate \textbf{D}iscrepancy  (\textbf{OED})' of the \texttt{LinReBoot} policy \eqref{eq:policy}. In principle,  the \textbf{OED} is given by
\begin{equation}
\textbf{OED} =
\text{Optimism} \times \texttt{Action Context Norm},
\end{equation}
where the \texttt{Action Context Norm} is given by $\norm{{\boldsymbol x}_{k}}_{{\boldsymbol V}_{t}^{-1}}$ and \text{Optimism} is given by $c_{t,k}$ for the $k$th action at time $t$, defined in \eqref{eq:colla_optimism}. Design of $c_{t,k}$ will be elaborated in Section \ref{sec:colla_opti}.

\textbf{Sufficient Explored Arms.}
We define the concept of \textit{Sufficient Explore Arms} to facilitate the formal regret analysis of \texttt{LinReBoot}. Intuitively, an arm is \textit{sufficient explored} if its index produced by the policy \eqref{eq:policy} is less than the mean reward of the optimal arm.
Technically, we say an arm $k$ is \textit{sufficiently explored} at time $t$ if the adopted OED ($c_{t,k}\norm{{\boldsymbol x}_{k}}_{{\boldsymbol V}_{t}^{-1}}$) is bounded by its optimal gap ($\Delta_{k}$).

The above notion of sufficient explored arm defines the concept of ''set of sufficient explored arms'' $\mathcal{S}_{t}$, formally
\begin{equation}\label{eq:SuffExpArms}
    {\cal S}_{t} := \{k\in [K]: c_{t,k} \norm{{\boldsymbol x}_{k}}_{{\boldsymbol V}_{t}^{-1}} < \Delta_{k}\},
\end{equation}
where and $c_{t,k}$ is the collaborated optimism and $c_{t,k}\norm{{\boldsymbol x}_{k}}_{{\boldsymbol V}_{t}^{-1}}$ is an optimistic estimate of discrepancy of policy index \eqref{eq:policy}.

The key consequence of set \eqref{eq:SuffExpArms} is that, any member in ${\cal S}_{t}$ enjoys the property
\begin{equation}
    \forall j \in {\cal S}_{t} \cap [K] : \tilde{\mu}_{j,t} < \mu_{1};
\end{equation}
that is, the \texttt{LinReBoot} policy always avoids an index \eqref{eq:policy} from sufficiently explored subset such that the bootstrapped mean of this index is less than the optimal mean reward unless all arm are sufficiently explored. (see equation \eqref{appendix: proof_lemma_gap_a} in the proof of Lemma \ref{appendix: lemma_gap} at section \ref{appendix: proof_lemma_gap} for technical details).

\subsection{Collaborate  Optimism} \label{sec:colla_opti}

Here we elaborate on the collaborated optimism adopted in the definition of sufficient explored arms \eqref{eq:SuffExpArms}.
Concretely, the collaborated optimism has a form
\begin{equation}
\label{eq:colla_optimism}
c_{t,k} = c_1(t,k) + c_{2}(t,k),
\end{equation} where $c_{1}(t,k)$ is called \textit{sample optimism} and $c_{2}(t,k)$ is called \textit{bootstrap optimism} for arm $k$ at time $t$.

\textbf{Sample Optimism.} The sample optimism $c_{1}(t,k)$ serves as a control on the event that ''the realized sample estimate discrepancy (ED)  is bounded by sample OED'':

\begin{subequations}
\label{event: sampling_concentration}
\begin{align}
    & E_{t,k} := \{|\Hat{\mu}_{k,t} - \mu_k| \leq c_{1}(t, k)  \norm{{\boldsymbol x}_{k}}_{{\boldsymbol V}_{t}^{-1}}, \}\label{eq:SamConcOneArm} \\
    & E_{t} := \bigcap_{k=1}^{K} E_{t,k},
    \label{eq:SamConcAllArm}
\end{align}
\end{subequations}
where $c_{1}(t, k)$ is a constant which can be tuned by our \texttt{LinReBoot} algorithm, making the bad event $\Bar{E}_{t,k}$ and $\Bar{E}$ become unlikely. In fact, this $E_{t,k}$ is the event that the least squared estimation is "close" to the true mean reward for arm $k$ at round $t$. In section \ref{sec:main_product}, the probability of the bad event $\Bar{E}_t$ is controlled by a parameter tuned by users based on lemma \ref{lemma: sampling_concentration}.

\textbf{Bootstrap Optimism.}

The bootstrap optimism $c_{2}(t,k)$ serves as a control on the event that ''the realized bootstrap ED is bounded by bootstrap OED'':
\begin{subequations}
\label{event: resampling_concentration}
\begin{align}
    & E_{t,k}^{\prime} := \{|\Tilde{\mu}_{k,t} - \Hat{\mu}_{k,t}| \leq c_{2}(t, k) \norm{{\boldsymbol x}_{k}}_{{\boldsymbol V}_{t}^{-1}} \},
    \label{eq:ReSamConcOneArm}\\
    & E_{t}^{\prime} := \bigcap_{k=1}^{K} E_{t,k}^{\prime},
    \label{eq:ReSamConcAllArm}
\end{align}
\end{subequations}
where $c_{2}(t, k)$ is also a constant controlling the conditional probability of the bad event $\Bar{E}_{t}^{\prime}$. This $c_{2}(t, k)$ can be tuned by our \texttt{LinReBoot} algorithm as well. Similar to $E_{t,k}$, this $E_{t,k}^{\prime}$ is the event that the residual bootstrap based estimation is "close" to the least squared estimate $\Hat{\mu}_{k,t}$ for arm $k$ at round $t$. In section \ref{sec:main_product}, the probability of bad event $\Bar{E^{\prime}_{t}}$ is controlled by a parameter tuned by users based on lemma \ref{lemma: resampling_concentration}.

\subsection{Optimism Design}

\textbf{Choice of sample optimism ($\alpha$).}
The goal of this part is to illustrate how to pick the sample OED such that the event \eqref{event: sampling_concentration} holds with probability at least $1-\alpha$ for a given confidence budget $\alpha \in (0,1)$.
Formally, the goal is to find a sample OED function $c_{1}(t, k):[n]\times [K] \mapsto  \mathbb{R}$ such that the event \eqref{eq:SamConcOneArm} holds with probability at least $1-\alpha_{k}$.
To meet the purpose of the risk control, we specify the sample OED function with form
\begin{equation}\label{eq:sample_OED}
    c_{1}(t, k):= R_2 \sqrt{d \log((1 + tL^{2}/\lambda)/\alpha_{k})} + \lambda^{1/2} S_2 .
\end{equation}
Lemma \ref{lemma: sampling_concentration} gives the formal result on why such choice has confidence budget at most $\alpha_{k}$. For regret analysis,  define $\alpha_{\min}= \underset{k \in [K]}{\min} \alpha_k $ and ${\boldsymbol \alpha} = (\alpha_1,...,\alpha_K)^{\top}$.

\textbf{Choice of bootstrap optimism ($\beta$).}
The goal of this part is to pick bootstrapped OED such that the event \eqref{event: resampling_concentration} holds with probability at least $1-\beta$ for given confidence budget $\beta \in (0,1)$.
Formally, the goal is to find a sample OED function $c_{2}(t, k):[n]\times [K] \mapsto  \mathbb{R}$ such that the event \eqref{eq:ReSamConcOneArm} holds with probability at least $1-\beta_{k}$.
To meet the purpose of the risk control, we specify the bootstrapped OED function with form
\begin{equation}\label{eq:bootstrap_OED}
c_{2}(t, k):=
\sqrt{
(2 \sigma_{\omega}^{2} RSS_{k,t} \log(2/\beta_{k}))
/
s_{k,t-1}^2 \norm{{\boldsymbol x}_{k}}_{{\boldsymbol V}_{t}^{-1}}^2
} .
\end{equation}

Lemma \ref{lemma: resampling_concentration} gives the formal result on why such choice has a confidence budget at most $\beta_{k}$. For regret analysis, let $\beta_{\min}$ be the smallest $\beta_k \mbox{, } \forall k \in [K]$ and ${\boldsymbol \beta} = (\beta_1,...,\beta_K)^{\top}$.

\subsection{Optimism for Optimal Arm}

\textbf{Sample-Bootstrap OED ratio of the optimal arm (b).} Indicated by the regret analysis in \citep{kveton2019perturbed}, instead of controlling the exploration independently, the relation between two sources of explorations needs to be considered because this relation is critical for finding the optimal action. To meet such observation, we define a good event,
\begin{equation}
\label{event: anti_concentration}
\begin{aligned}
    E_{t}^{\prime \prime} :=
    \{\Tilde{\mu}_{1,t} - \Hat{\mu}_{1,t} > c_{1}(t, 1) \norm{{\boldsymbol x}_{1}}_{{\boldsymbol V}_{t}^{-1}} \}.
\end{aligned}
\end{equation}
Given the good event $E_{t}^{\prime \prime}$, the policy index $\Tilde{\mu}_{1,t}$ of the optimal arm enjoys further positive bias, hence the agent will have better chance to make optimal action.

In particular, we highlight a constant $b$ used to measure the ratio of the sample optimism \eqref{eq:sample_OED} to the bootstrap optimism \eqref{eq:bootstrap_OED}; formally, we require $b$ satisfies
\begin{equation}\label{eq:Samp-Boot_ratio}

    c_{1}(t, 1)/c_{2}(t, 1) \ge b \cdot \sqrt{2 \log \left(2/\beta_{1}\right)}.
\end{equation}
Intuitively, the constant $b$ measures the relation between sample OED and bootstrap OED of the optimal arm. This $b$ plays an important role of the probability lower bound of event \eqref{event: anti_concentration} (See Lemma \ref{lemma: anti_concentration}). Note that,
if \eqref{eq:Samp-Boot_ratio} holds, we have the lower bound \eqref{eq:ant1}
; otherwise, we have the lower bound \eqref{eq:ant2}. In both cases, we have a lower bound for the event \eqref{event: anti_concentration}.

\textbf{Good event for optimal arm ($\gamma$).} Here we introduce the event that over exploration and under exploration of the optimal arm have been avoided simultaneously. Formally, the constant $\gamma$ is the probability that the bandit index \eqref{eq:policy} is not over-exploration (Event $E_{t}^{\prime}$) and also not under-exploration (Event $E_{t}^{\prime \prime}$)
\begin{equation}\label{eq:not_over_under}
\{c_{1}(t, 1)  <
(\Tilde{\mu}_{1,t} - \Hat{\mu}_{1,t})
/
\norm{{\boldsymbol x}_{1}}_{{\boldsymbol V}_{t}^{-1}}
< c_{2}(t, 1)  \}.
\end{equation}
Technically, we can show that the probability of the event \eqref{eq:not_over_under} is lower bounded by the term
\begin{equation}
\mathbb{P}_{t}(E_{t}^{\prime \prime}) - \mathbb{P}_{t}(\Bar{E}_{t}^{\prime}),
\end{equation}
with probability at least $1-\gamma$ (Lemma \ref{lemma: connection_concentrations}).
Such lower bound is translated into an upper bound in regret analysis.

\section{Formal Results}\label{sec:main_product}

\begin{table}[t!]
\centering
\renewcommand{\arraystretch}{1.75}

\begin{tabular}{ c|c }
\hline
Notation & Definition \\
\hline

\multirow{2}{*}{$\zeta_1 (n,d)$} &
$
(L_2 \sqrt{d \log(\frac{1 + n L^{2}/\lambda}{\alpha_{\min}})} + \lambda^{1/2} S_2)
\times $ \\
 &
$
\sqrt{2(n-K) d \log(1 + \sum_{i=1}^r \sigma_i^2/d \lambda )}
$ \\
\hline

\multirow{2}{*}{$\zeta_2 (n,d)$} &
$
\sqrt{2 \sigma_{\omega}^{2} log(\frac{2}{\beta_{\min}})} \times
$  \\
 &
$
\sqrt{2(n-K) d \log(1 + \sum_{i=1}^r \sigma_i^2/d \lambda )}
 $ \\
\hline

$\zeta_3(n)$
&  $2 K \sqrt{4 L_2\sigma_{\omega}^{2} \log(\frac{2}{\beta_{\min}})}
(\log n + 1)$ \\
\hline

$\zeta_4(n)$
&
$
2 S_2 L   ((n - K) (\alpha + \beta) + K - 1)
$\\
\hline
\end{tabular}

\caption{\footnotesize Notations in Regret Analysis}
\label{table: regret_bound_n}

\end{table}

\subsection{Regret Bound for \texttt{LinReBoot}}
\label{subsec: regret_bound}

\begin{theorem}
\label{theorem: main}
Under Assumptions \ref{ass:bound}, \ref{ass:noise_bound}, \ref{ass: lower_bound} and technical conditions \eqref{appendix: proof_theorem_5.1_rho} and \eqref{requirement_for_b},  with probability at least $1-(\delta + \gamma)$, the expected regret of Algorithm \ref{alg:LinReBoot: Version_1}  is bounded as,
\begin{equation}
\begin{aligned}
    R_{n}
    \leq
    & C_1 (\alpha_1, {\boldsymbol \beta}, \gamma, b) \zeta_1 (n,d) \\
    +
    & C_2 ({\boldsymbol \alpha}, {\boldsymbol \beta}, \gamma, b, \delta) \zeta_2 (n,d) \\
    +
    & C_1 (\alpha_1, {\boldsymbol \beta}, \gamma, b) \zeta_3 (n) +
    \zeta_4 (n),
\end{aligned}
\end{equation}
where $\zeta_1$, $\zeta_2$, $\zeta_3$ and $\zeta_4$ are defined in Table.\ref{table: regret_bound_n} and $C_1 $, $C_2$, $M_1$, $M_2$ are described in Table.\ref{table: regret_bound_constants}.
\end{theorem}
\begin{proof}

See Appendix \ref{appendix: proof_theorem_main}.

\end{proof}

\begin{corollary}
\label{corollary: rate}
Let ${\boldsymbol \alpha} = {\boldsymbol \beta} = \frac{1}{\sqrt{n}}{\boldsymbol 1}$, the order of high probability upper bound in Theorem \ref{theorem: main} is $\Tilde{O}(d \sqrt{n})$.
\end{corollary}
\begin{proof}

See Appendix \ref{appendix: proof_corollary_rate}.

\end{proof}
Corollary \ref{corollary: rate} shows that our regret bound scales as the regret bound of Linear Thompson sampling \citep{agrawal2013thompson} and Linear PHE \citep{kveton2019perturbed}.

\begin{figure*}[th!]
\centering

\includegraphics[scale = 0.2]{figures/Summary.png}

\caption{Comparison of \texttt{LinReBoot} with Gaussian Bootstrap weights to baselines under three linear bandit problems and three different context dimension $d$. First row referred to the setting in Section \ref{subsec: SLB_experiment}, second row is for Section \ref{subsec: LB_random_experiment} and the last row is for Section \ref{subsec: LB_covariates_experiment}. Three columns refer to $d=5$, $d=10$ and $d=20$ respectively.}
\label{fig: summary}

\end{figure*}

\subsection{Validate Sample Optimism}
\label{subsec: sampling_concentration}

\begin{lemma}
\label{lemma: sampling_concentration}
Under Assumptions \ref{ass:bound}, \ref{ass:noise_bound}, \ref{ass: lower_bound} and choose $c_{1}(t,k)$ as \eqref{eq:sample_OED},  $\mathbb{P}(\Bar{E}_{t,k})$, the probability of bad event corresponded to least squared estimation described in (\ref{event: sampling_concentration}), is controlled. Formally, $\forall k \in [K]$, $\forall \alpha_{k} > 0$, $\forall t \geq 1$,
\begin{equation}
    \mathbb{P}( |\Hat{\mu}_{k,t} - \mu_{k}|
    \leq
    c_{1}(t, k)\norm{{\boldsymbol x}_{k}}_{{\boldsymbol V}_{t}^{-1}} )
    \geq
    1 - \alpha_{k}.
\end{equation}
Consequently, we have $\mathbb{P}(\Bar{E}_t) \leq \alpha := \sum_{k=1}^{K} \alpha_{k}$.
\end{lemma}
\begin{proof}

See Appendix \ref{appendix: proof_lemma_sampling_concentration}.

\end{proof}
Lemma \ref{lemma: sampling_concentration} supports that the choice  of $c_{1}(t,k)$ at \eqref{eq:sample_OED} for the sample optimism event \eqref{event: sampling_concentration} is valid with confidence budget $\alpha$.

\subsection{Validate Bootstrap Optimism}
\label{subsec: resampling_concentration}

\begin{lemma}
\label{lemma: resampling_concentration}
Suppose bootstrap weights are Gaussian. Pick $c_{2}(t,k)$ as \eqref{eq:bootstrap_OED}. The conditional probability of bad event corresponding to residual bootstrap exploration described in \eqref{event: resampling_concentration}, $\mathbb{P}_t(\Bar{E}_{t,k}^{\prime})$, is controlled. Formally,
$\forall k \in [K]$, $\forall \beta_{k} > 0$, $\forall t \geq 1$
\begin{equation}
    \mathbb{P}_{t} (|\Tilde{\mu}_{k,t} - \Hat{\mu}_{k,t}|
    \leq
    c_{2}(t, k)\norm{{\boldsymbol x}_{k}}_{{\boldsymbol V}_{t}^{-1}})
    \geq
    1 - \beta_{k} .
\end{equation}
Consequently, we have $\mathbb{P}_{t}(\Bar{E^{\prime}_{t}}) \leq \beta := \sum_{k=1}^{K} \beta_{k}$.
\end{lemma}

\begin{proof}
See Appendix \ref{appendix: proof_lemma_resampling_concentration}.
\end{proof}

Lemma \ref{lemma: resampling_concentration} supports that the choice  of $c_{2}(t,k)$ at \eqref{eq:bootstrap_OED} for the sample optimism event \eqref{event: resampling_concentration} is valid with confidence budget $\beta$.

\subsection{Sample-Bootstrap ratio}
\label{subsec: anti_concentration}

\begin{lemma}
\label{lemma: anti_concentration}
Under Assumptions \ref{ass:bound}, \ref{ass:noise_bound}, \ref{ass: lower_bound}. Suppose bootstrap weights are Gaussian. The conditional probability of anti-concentration for optimal arm described in (\ref{event: anti_concentration}), $\mathbb{P}_t(\Bar{E}_{t}^{\prime \prime})$, has lower bound. Formally, if $b$ satisfies \eqref{eq:Samp-Boot_ratio},
\begin{equation}
\begin{aligned}
    \mathbb{P}_{t}(E^{\prime \prime}_{t})
    \geq
    \frac{b}{\sqrt{2 \pi}}
    \exp(-\frac{3 c_{1}^2(t,1) s_{1, t-1}^2 \norm{{\boldsymbol x}_{1}}_{{\boldsymbol V}_{t}^{-1}}^2}{ 2 \sigma_{\omega}^2 RSS_{1,t}}) .
    \label{eq:ant1}
\end{aligned}
\end{equation}
Otherwise,
\begin{equation}
\begin{aligned}
    \mathbb{P}_{t}(E^{\prime \prime}_{t})
    \geq
    \Phi(-b)
    \label{eq:ant2},
\end{aligned}
\end{equation}
where $\Phi$ is the CDF of standard normal distribution.
\end{lemma}
\begin{proof}

See Appendix \ref{appendix: proof_lemma_anti_concentration}.

\end{proof}

Lemma \ref{lemma: anti_concentration} provides the lower bound result for good event $E_{t}^{\prime \prime}$. The result indicates that, if the bootstrap optimism is not 'too large', then the \texttt{LinReBoot} procedure can enjoy additional regret reduction.

\subsection{Validate good event}

\label{subsec: connection_concentrations}
\begin{lemma}
\label{lemma: connection_concentrations}
Under Assumptions \ref{ass:bound}, \ref{ass:noise_bound}, \ref{ass: lower_bound} and suppose Bootstrap weights are Gaussian. Assume $b$ satisfies a technical condition \eqref{requirement_for_b}. Then, with probability at least $1- \gamma$, $\mathbb{P}_{t}(E_{t}^{\prime \prime}) - \mathbb{P}_{t}(\Bar{E}_{t}^{\prime})$ has lower bound,
\begin{equation}
\begin{aligned}
\dfrac{b}{\sqrt{2 \pi}} \exp( - \frac{3 s_{1, t-1}^{3/2} c_{1}^{2}(t,1) \norm{{\boldsymbol x}_{1}}^{2}_2 }{8 \sigma^{2}_{\omega} (\sigma_{\min}^2 + \lambda) \sqrt{\frac{1}{M_2} \log(\frac{M_1}{1 - \gamma})}}) -\beta,
\end{aligned}
\end{equation}
where $M_1$ and $M_2$ are defined in Table.\ref{table: regret_bound_constants}.
\end{lemma}
\begin{proof}

See Appendix \ref{appendix: proof_lemma_connection_concentrations}.

\end{proof}

Lemma \ref{lemma: connection_concentrations} provided the a high probability lower bound for the difference between probability of the event for anti-concentration $E_{t}^{\prime \prime}$ and probability of bad event discussed in bootstrap optimism in Section \ref{sec:colla_opti}. This lower bound is also for probability of `not under and not over exploration' event \eqref{eq:not_over_under}. Lemma \ref{lemma: connection_concentrations} links the sample optimism and bootstrap optimism and holds a right amount of exploration of the optimal arm.

\section{Experiments}\label{sec:exp_to_imple}

In this section, we conduct empirical studies under three settings: Stochastic Linear Bandit, Contextual Linear Bandit and Linear Bandit with Covariates. Our \texttt{LinReBoot} is compared to several baselines including \texttt{LinTS-G} \citep{agrawal2013thompson, lattimore2020bandit}, \texttt{LinTS-IG} \citep{honda2014optimality, riquelme2018deep}, \texttt{LinPHE} \citep{kveton2019perturbed}, \texttt{LinGIRO} \citep{kveton2019garbage} and \texttt{LinUCB}  \citep{abbasi2011improved, lattimore2020bandit} . More details about baselines can be found in Appendix \ref{appendix: experiment_other_algs}.

\subsection{Stochastic Linear Bandit}
\label{subsec: SLB_experiment}

We compare \texttt{LinReBoot} to other linear bandit algorithms under stochastic linear bandit described in Section \ref{sec: Stochastic_Linear_Bandit}.
We experiment with several dimensions $d$ including $5$, $10$ and $20$. $K$ is chosen as $100$. Synthetic data generation for this setting is deferred to Appendix \ref{appendix: experiment_settings} in the supplementary material.
\textbf{Results.} The first row of Figure \ref{fig: summary} reports the results for Stochastic Linear Bandit setting.
Our \texttt{LinReBoot} rivals \texttt{LinTS-G} and \texttt{LinTS-IG} while substantially exceeds \texttt{LinGIRO}, \texttt{LinPHE} and \texttt{LinUCB}. When $d$ increases, the performance of \texttt{LinReBoot} rivals and exceeds the best of other methods.

\subsection{Contextual Linear Bandit}
\label{subsec: LB_random_experiment}

In the second experiment, we compare \texttt{LinReBoot} to other linear bandit algorithms under Contextual Linear Bandit where the contexts are generated from some distributions by arms. Note that this setting matches previous work \citep{chu2011contextual}. Linear bandit algorithms can also be applied under this kind of environment. In our experiment, the \texttt{LinReBoot} is implemented as Algorithm \ref{alg:LinReBoot: Version_2} in Appendix \ref{appendix: experiment_algs}. Like the setting in Section \ref{subsec: SLB_experiment}, the dimension of $d$ is chosen as $5$ or $10$ or $20$ and the synthetic data generation for this setting is described in Appendix \ref{appendix: experiment_settings}.
\textbf{Results.} The second row of Figure \ref{fig: summary} reports the results for Contextual Linear Bandit. Our \texttt{LinReBoot} rival \texttt{LinTS-G} and substantially exceed \texttt{LinTS-IG}, \texttt{LinGIRO}, \texttt{LinPHE} and \texttt{LinUCB}. When $d$ increases, the performance of \texttt{LinReBoot} rivals \texttt{LinTS-IG} and exceeds others.

\subsection{Bandit with Covariates}
\label{subsec: LB_covariates_experiment}

Our last experiment is conducted under the setting of linear bandit with covariates, which is also called linear parametrized bandit by \citep{rusmevichientong2010linearly}. This problem is significantly different from the previous two problems in the following ways. Each arm has its true parameter ${\boldsymbol \theta}_k$.  That is, each arm has its estimate $\Hat{{\boldsymbol \theta}}_k$ from the ridge regression procedure in Section \ref{subsec: LinReBoot}. Also, unlike the setting in Section \ref{subsec: LB_random_experiment}, the contexts are generated from a distribution that is independent of arms. Thus the overall task in this setting is not only the estimation of the target parameter ${\boldsymbol \theta}$, but also the detection of which arm a context belongs to. This case is also referred to as the online decision-making under covariates \citep{bastani2020online}. For the \texttt{LinReBoot} in this setting, detailed algorithm is provided as Algorithm \ref{alg:LinReBoot: Version_3} in Appendix \ref{appendix: experiment_algs}. $d$ is chosen as $5$ or $10$ or $20$ and $K=10$. Synthetic data generation for this setting is described in Appendix \ref{appendix: experiment_settings}.
\textbf{Results.} The third row of Figure \ref{fig: summary} reports the results for Linear Bandit with Covariates. Our \texttt{LinReBoot} exceeds all competing algorithms \texttt{LinTS-G}, \texttt{LinTS-IG}, \texttt{LinGIRO}, \texttt{LinPHE} and \texttt{LinUCB}.

\textbf{Summary.}
From Figure \ref{fig: summary}, the proposed \texttt{LinReBoot} is always the top 3 algorithms under all settings and all choice of dimension $d$. More specifically, \texttt{LinReBoot} is clearly comparable to the state-of-the-art Linear Thompson Sampling algorithms(\texttt{LinTS-G}, \texttt{LinTS-IG}) or even outperforms them in many cases. Regarding the computational cost, from Table.\ref{table: comp_cost}, our proposed \texttt{LinReBoot} is consistently computational efficient among all settings compared to \texttt{LinTS-G}, \texttt{LinTS-IG} and \texttt{LinUCB} under all three settings.

\section{Conclusion}\label{sec:DisConclu}

We propose \texttt{LinReBoot} algorithm for stochastic linear bandit problems. In theory, we prove \texttt{LinReBoot} that secures $\tilde{O}(d \sqrt{n})$ high probability expected regret. Empirically, we show \texttt{LinReBoot} rivals \texttt{LinTS-G}, \texttt{LinTS-IG} and exceeds  \texttt{LinPHE}, \texttt{LinGIRO} and \texttt{LinUCB}, which supports the easy-generalizability of \texttt{ReBoot} principle in \citep{wang2020residual} under various contextual bandit settings including Stochastic Linear Bandit, Contextual Linear Bandit, and Linear Bandit with Covariates.

\clearpage

\vskip 0.2in
\nocite{*}

redpajama_set_name: RedPajamaArXiv


I believe in progress. In recognizing and acknowledging your growth. In appreciating not just everything that you have but also everything you've endured to get to where you are. In process. In stopping to consider where you are, not just where you are trying to go. In gratitude and contentment. In looking back only to be reminded of where you've come from. To see the lessons and to teach them and to inspire. To never forget where you came from. Your roots. Your experiences. Everything that made you who you are today. Everything that built you. You are not just the positive. You are the negative too. The struggle. The hustle. The grime. The beautiful and the broken. The good and the bad. You are balance. I regret nothing. There's no reason to. I am not just made up of the good things. I've stolen. I've lied. I've been drunk. I've been hungry. I've been homeless. I've been hopeless. But I've also been happily married for seven years to a man I've been with ten years. I've also produced documentaries and taught women and children all over the world, hosted radio shows, performed poetry, taught grown women how to read, fed the homeless, helped felons to find jobs, and authored nine books and counting. I am not perfect, trust I have my days. Some are harder than others and I wonder how I'll make it through but then, I believe in progress.
Yecheilyah (e-see-lee-yah) is an Author, Blogger, and Poet of nine published works including work in progress and short inspirational guide "Keep Yourself Full." Learn more by exploring Yecheilyah's writing on this blog and her website at yecheilyahysrayl.com.
Renaissance: The Nora White Story (Book One) is her latest novel and is available now on Amazon.com.


text:
Redding Colt 45s pound out 13 hits in Sunday win
Mike Chapman
The Redding Colt 45s salvaged one game out of a three-game series against the Humboldt B52s with a 12-3 victory Sunday at Tiger Field.
The Colt 45s were as hot as Sunday's 103-degree game-time temperature as they pounded out 13 hits.
Jake Gozzo sent Redding to an early lead with a three-run home run in the bottom of the first inning. Gozzo finished the day by going 3 for 4 with four RBIs and two runs.
Coby Zimmerman also hit a home run — a solo shot in the third. Other top Colt 45s hitters were Derick Kuhn, Eric Carlson and Mickey Meinhoefer, who each went 2 for 5. Kuhn had three runs while Carlson and Meinhoefer had a run apiece.
RELATED: Home runs highlight Colt 45s' weekend series
JJ Reeves was the winning pitcher, giving up one run on three hits and a walk while striking out six over five innings.
The Colt 45s lost 12-4 on Saturday and also were defeated 3-2 Friday.
Starter James Humphrey was Saturday's losing pitcher. He gave up six runs on four hits and two walks over 1.1 innings. Teammates Gozzo went 3 for 4, Max Werner was 2 for 4 with a run and Lane Pritchard was 2 for 5 with a run.
RELATED: Colt 45s riding a four-game winning streak; Crabs up next
Andrew Eppenbach took the loss Friday. Gozzo hit one out of the park and Carlson was 2 for 3.
The Colt 45s (9-9) host the San Leandro Pilots for a two-game set starting at 7:15 p.m. Wednesday.
RELATED: Colt 45s fall to Seattle Studs at Tiger Field
Far Out Friday
Friday's game against the Redding Ringtails will be held on '60s Night. Spectators who wear a tie-dye T-shirt will get $1 off a $5 general admission ticket. The first pitch is at 7:15 p.m. with gates opening at 6:30 p.m.


Solution: (c) In a hydrogen atom, electron revolving around a fixed proton nucleus have some centripetal acceleration. Therefore its frame of reference is non- inertial. If the frame of reference, where the electron is at rest, the given expression is not true as it forms the non-inertial frame of reference.
Solution: (a) The simple Bohr model cannot be directly applied to calculate the energy levels of an atom with many electrons because when we derive the formula for radius/energy levels etc, we make the assumption that centripetal force is provided only by electrostatic force of attraction by the nucleus. Hence, this will only work for single electron atoms. In multi-electron atoms, there will also be repulsion due to other electrons. The simple Bohr model cannot be directly applied to calculate the energy levels of an atom with many electrons.
Bohr's model gives only the magnitude of angular momentum. The angular momentum is a vector quantity. Hence we cannot express angular momentum completely by Bohr model. Hence it does not give correct values of angular momentum of revolving electron.
Key concept: Forces that keep the nucleons bound in the nucleus are called nuclear forces.
• Nuclear forces are short range forces. These do not exist at large distances greater than 10~15 m.
• Nuclear forces are the strongest forces in nature.
• These are attractive force and causes stability of the nucleus.
• These forces are charge independent.
• Nuclear forces arc non-central force.
The nuclear binding force has to dominate over the Coulomb repulsive force between protons inside the nucleus. The nuclear force between two nucleons falls rapidly to zero as their distance is more than a few femtometres. .
In O2 molecule which consists of two oxygen atoms molecules, nuclear force between the nuclei of the two atoms is not important because nuclear forces are short-ranged and act inside the nucleus only.
(d) to emit photons only when they collide .
Solution: (a, c) In a hydrogen atom, electron revolves around a fixed proton nucleus in circular path. This can be explained by Bohr model. But in case of ionised H-molecule which consists of two protons in nucleus and where protons are separated by a small distance of the order of angstrom, cannot be explained by Bohr model. Hence in this case the ground state the electron would not move in circular orbits, the electrons orbit would go around the protons.
Question 9. Consider aiming a beam of free electrons towards free protons. When-they scatter, an electron and a proton cannot combine to produce a H-atom.
Solution: (a, b) A moving electron and proton cannot combine to produce a H-atom because of energy conservation and without simultaneously releasing energy in the form of radiation.
Solution: (a, b) Bohr proposed a-model for hydrogen atom which is also applicable for some lighter atoms in which a single electron revolves around a stationary nucleus of positive charge Ze (called hydrogen like atom, e.g.: H, He+, Li+2, Na+1 etc). It is not applicable to hydrogen in the molecular form and also, it will not be applicable as it is for a He-atom.
(c) in any transition in a H-atom .
From above discussion we can say Balmer series for the H-atom can be observed if we measure the frequencies of light emitted due to transitions between higher excited states and the first excited state and as a sequence of frequencies with the'higher frequencies getting closely packed.
Solution: (b, d) Let E2 and E1 be the energy corresponding to n = 2 and n = 1 respectively. If radiation of energy ∆E = (E2 – E1) = hf incident on a sample where all the H-atoms are in the ground state, according to Bohr model some of the atoms will move to the first excited state. As this energy is not sufficient for transition from n = 1 to n =3, hence no atoms will make a transition to the n = 3 state.
Key concept: Bohr proposed a model for hydrogen atom which is also applicable for some lighter atoms in which a single electron revolves around a stationary nucleus of positive charge Ze (called hydrogen like atom). It is valid only for one electron atoms, e.g. : H, He+, Li+2, Na+1 etc.
The Bohr model is not applicable to He4, as it has one more electron and electrons are not subjected to central forces.
Question 14. The mass of a H-atom is less than the sum of the masses of a proton and electron. Why is this?
Question 15. Imagine removing one electron from He4 and He3. Their energy levels, as worked out on the basis of Bohr model will be very close. Explain why.
Solution: Bohr model is applicable for hydrogen atom-and some lighter atoms in which a single electron revolves around a stationary nucleus of positive charge Ze (called hydrogen like atom). If we remove one electron from He4 and He3, atoms contain one electron and becomes hydrogen like atoms. Now we can apply Bohr model to define the energy levels.
difference in the energies appears in the form of electromagnetic radiation. Why cannot it be emitted as other forms of energy? .
Solution: The electrons are charged particles. When an electron falls from a higher energy to a lower energy level, it accelerates. We know accelerating charged particle radiates energy in the form of electromagnetic radiation.
Question 17. Would the Bohr formula for the H-atom remain unchanged if proton had a charge (+ 4/3)e and electron a charge (-3/4)e, where e = 1.6 x 10-19 C. Give reasons for your answer.
Solution: According to Bohr, for an electron around a stationary nucleus the electrostatics force of attraction provides the necessary centripetal force.
If proton had a charge (+4/3)e and electron a charge (-3/4)e, then the Bohr formula for the H-atom remains same, since the Bohr formula involves only the product of the charges which remain constant for given values of charges.
Question 18. Consider two different hydrogen atoms. The electron in each atom is in an excited state. Is it possible for the electrons to have different energies but the same orbital angular momentum according to the Bohr model?
Solution: Bohr postulated that an electron in an atom can move around the nucleus in a certain circular stable orbits without emitting radiations.
Question 19. Positronium is just like a H-atom with the proton replaced by the positively charged anti-particle of the electron (called the positron which is as massive as the electron). What would be the ground state energy of positronium?
Key concept: Positronium (Ps) is a system consisting of an electron and its anti-particle a positron, bound together into an exotic atom, specifically anonium. The system is unstable: the two particles annihilate each other to predominantly produce two or three gamma-rays, depending on the relative spin states. The orbit and energy levels of the two particles arc similar to that of the hydrogen atom (which is a bound slate of a proton and an electron). I lowever, because of the reduced mass, the frequencies of the spectral lines are less than half of the corresponding hydrogen lines.
Question 20. Assume that there is no repulsive force between the electrons in an atom but the force between positive and negative charges is given by Coulomb's law as usual. Under such circumstances, calculate the ground state energy of a He-atom.
Question 21. Using Bohr model, calculate the electric current created by the electron when the H-atom is in the ground state.
Question 22. Show that the first few frequencies of light that is emitted when electrons fall to nth level from levels higher than n, are approximate harmonics (i.e., in the ratio 1:2:3 …) when n >> 1.
Question 23. What is the minimum energy that must be given to a H-atom in ground state so that it can emit an Hґ Line in Balmer series? If the angular momentum of the system is conserved, what would be the angular momentum of such Hґ photon?
Question 24. The first four spectral in the Lyman series of a H-atom are λ= 1218 Å, 1028 Å, 974.3 Å and 951.4 Å. If instead of Hydrogen, we consider deuterium, calculate the shift in the wavelength of these lines.
Solution: Let µH and µD are the reduced masses of electron for hydrogen and deuterium respectively.
Question 25. Deutrium was discovered in 1932 by Harold Urey by measuring the small change in wavelength for a particular transition in 1H and 2H. This is because, the wavelength of transition depend to a certain extent on the nuclear mass. If nuclear motion is taken into account, then the electrons and nucleus revolve around their common centre bf mass.
Question 26. If a proton had a radius R and the charge was uniformly distributed, calculate using Bohr theory, the ground state energy of a H-atom when (i) R = 0.1 Å and (ii) R = 10 Å .
Solution: In a H-atom in ground state, electron revolves round the point-size proton in a circular orbit of radius rB (Bohr's radius).
Question 27. In the Auger process, an atom maizes a transition to a lower state without emitting a photon. The excess energy is transferred to an outer electron which may be ejected by the atom (This is called an Auger electron). Assuming the nucleus to be massive, calculate the kinetic energy of an n = 4 Auger electron emitted by Chromium by absorbing the energy from a n – 2 to n = 1 transition.
Auger Effect: The Auger effect is a process by which electrons with characteristic energies are ejected from atoms in response to a downward transition by another electron in the atom. In Auger spectroscopy, the vacancy is produced by bombardment with high energy electrons, but the Auger effect can occur if the vacancy is produced by other interactions. It is observed as one of the methods of electron rearrangement after electron capture into the nucleus.
If an inner shell electron is removed from an atom, an electron from a higher level will quickly make the transition downward to fill the vacancy. Sometimes this transition will be accompanied by an emitted photon whose quantum energy matches the energy gap between the upper and lower level. Since for heavy atoms this quantum energy will be in the x-ray region, it is commonly called x.-ray fluorescence. This emission process for lighter atoms and outer electrons gives rise to line spectra.
Sometimes an upper election drops to fill the vacancy, emitting a photon.
As the nucleus is massive, recoil momentum of the atom may be neglected and the entire energy of the transition may be considered transferred to the Auger electron. As there is a single valence electron in Cr, the energy states may be thought of as given by the Bohr model.


text:
Ritchie Elkins
Ritchie Elkins, 82 of Greensburg, passed away on March 25, 2020 in Connersville. He was born on March 2, 1938 in Jackson County, IN to the late Claude and Ruth (McKain) Elkins. Ritchie was a 1956 graduate of Kokomo High School. He married Rosemary (Tolly) Moffett Elkins on October 5, 2010 and she survives. Ritchie served his country in the United States National Guard. He managed Good Year Tire Store for several years and after retirement drove a semi. Ritchie was an avid golfer and fisherman. In addition to his wife, Rosemary, he is survived by his children, Kelly Sandy of Indianapolis, Jeff (Heather) Elkins of Greencastle, IN, and Brent (Mary) Elkins of, FL and step children, Jeannie (John) Thoman of Rush County, John D. (Linda) Moffett of Rushville, Jim (Patty) Moffett of Rushville, Dennis (Michelle) Moffett of Greensburg, and Richard Moffett of Rushville; and several grandchildren and great-grandchildren; brother, Dennis (Merle) Elkins. Celebration of life will be held at a later date for Ritchie. Memorial contributions may be made out to Alzheimer's Association. Moster Mortuary has been entrusted to the arrangements.
Ritchie Elkins, 82 of Greensburg, passed away on March 25, 2020 in Connersville. He was born on March 2, 1938 in Jackson County, IN to the late Claude and Ruth (McKain) Elkins. Ritchie was a 1956 graduate of Kokomo High School. He married Rosemary... View Obituary & Service Information
The family of Ritchie Elkins created this Life Tributes page to make it easy to share your memories.
Ritchie Elkins, 82 of Greensburg, passed away on March 25, 2020...
Send flowers to the Elkins family.


IPsoft is looking for an exceptional Analyst Relations Manager to join its AR team.
Amelia, their industry-leading cognitive agent, is an artificial intelligence platform that makes it possible to automate knowledge work across a broad range of functions. Exposed to the same information as any new hire, she can quickly apply her knowledge to solve queries in a wide range of business processes. With Amelia able to shoulder the burden of tedious, often laborious tasks, she partners with human coworkers to achieve new levels of productivity and service quality.
• Facilitate the timely completion of all relevant analyst briefing and survey requests. This includes awareness of analyst research publication and analyst firm event schedules.
• Contribute to regular status reports to leadership to demonstrate progress in the IAR program.
• Work with practice leaders to secure analyst opportunities and referrals and to generate positive analyst evaluations and reports.
• Interpersonal communications: Strong relationship management skills. Ability to build a rapport with internal and external clients. Ability to interact effectively with partners and senior management, as well as with peers and other colleagues.
• Ability to handle multiple projects and high‐level responsibilities simultaneously.


The Major Arcana of the Rider Waite Deck, along with it's attendant Minor, can serve to give us some guidance.
The Hanged Man dangles from his tree, his hands hidden from view, one leg crossed behind the other.
When you've found your heart's philosophy and truth, this is when Judgment will enter your life.
The Sun card from the Rider Waite deck burns away all questions, leaving only well understood answers.


text:
Q: How do I enable TLS 1.3 in iOS and MacOS? It's reported that TLS 1.3 (currently in draft status) can reduce connection setup delays by 30% or approximately 30ms.
How can I configure this on MacOS or iOS?

A: According to WWDC2017:

*

*TLS 1.3 is not enabled by default

*You can enable this on iOS with this provisioning profile

*You can enable this on MacOS with this command in Terminal
sudo defaults write /Library/Preferences/com.apple.networkd tcp_connect_enable_tls13 1

redpajama_set_name: RedPajamaStackExchange


The Garden-based Schools Programme offers environmental education lessons on biodiversity, ecology, climate change, sustainability, etc., to learners from grade R to grade 12. All the lessons are linked to the National Curriculum Statement and are facilitated by our trained Assistant Environmental Education Officers.
Our lessons are offered in English and/ or isiZulu depending on the learners' medium of instruction and the educators' language preferences.
As part of the garden-based school programme, a Biodiversity Careers Expo, where learners are given the opportunity to explore careers in the biodiversity field, is hosted by the education centre once every year, usually in February.
Environmental education lessons are also offered to out-of-school youth and community members on biodiversity conservation, indigenous garden development and climate change under the community outreach programme. The lessons are offered in IsiZulu or English. Lessons are offered both in the Garden and where beneficiaries are based.
The education centre is always exploring new ways to better its services for the benefit of learners and the community. One way we are doing this is by teaching learners and members of local communities how to evaluate the quality of the river water in their communities through MiniSass. The MiniSass method can be used by any non-technical person to monitor the health of a river. The education centre is the ideal environment to learn about this exciting method. Schools are encouraged to take this multi-sensory learning opportunity and experience edutainment and the beauty of outdoor learning.
Schools are requested to contact Lizeka Gqumani or Dr. Jabulani Mpungoseto book lessons at least one week in advance.