\section{Introduction}

In web services, users are often faced with a task of selecting an item from a large catalog. Examples include listening to a song from a catalog of 11 million on a music service or watching a video from 500 million user-uploaded videos on a video service. Equally challenging is the decision faced by the web service, the one hosting and presenting the choices to its users. Given limited screen real-estate, which of the 11 million songs or 500 million videos should it present to its users? Personalization has served as the de facto solution to this problem. Web services use proactive and reactive personalized recommendations to guide users to items that are relevant as well as help them discover new items they will also enjoy. From a mix of implicitly learned and explicitly collected features of each user and item, the services prune the list of available options that they present to each user and provide a personalized experience whilst doing so. While there are often competing objectives when deciding the optimal item or set of items to present to a user, overall, the goal is to satisfy the user in order to increase engagement with the service and retain the user over time.
For this purpose, recommendation engines in industry rely on \emph{multi-armed bandit} algorithms to learn how to optimally recommend items to the users. In particular \emph{contextual multi-armed bandits}, which attempt to learn the optimal personalized recommendation for each user given user information, i.e., the context, are widely used in practice.

While personalization in web services makes the catalog more accessible to a user by reducing the burden of choice, it has the potential to isolate the user and unintentionally create ``filter bubbles''. This may happen even if the users are modeled by the service only using implicit behavioral data, since the feedback loops of online learning
can cause an increased focus on narrower interests.
This has the potential of hindering a user from participating
in social conversation within their network on specific content, as the content that is served to different individuals in the network may differ drastically.
Additionally, as the user context gets more detailed and higher-dimensional, the model estimation of a contextual bandit becomes more challenging and the regret bounds may take longer to converge.
\cite{dimakopoulou2018estimation} have shown that all else equal, using assignment policies that are simpler (in terms of how they vary with contextual variables) in the early learning phases of the algorithm can improve the rate of learning and decrease regret.
Finally, simpler, unpersonalized assignment rules may have other advantages as well; for example, \cite{lei2017actor} highlight the advantages of simplicity for interpretability in health applications of contextual bandits.

We design a new family of bandit algorithms that interpolate between unpersonalized and personalized recommendations.
This new family of algorithms, called \emph{selectively contextual bandits} (SCB), chooses between a contextual bandit decision and a non-contextual bandit decision in every iteration. The context is only used when the contextual decision yield a predicted reward lift higher than a parameter \begin{math}\delta\end{math}.
The \begin{math}\delta\end{math} parameter is annealed at a rate that depends on the regret bound of the upper confidence bound (UCB) such that we optimize the regret bound to remove dependence on the dimensionality of the context. Alternatively, we can regularize the estimator of Lin-UCB towards a non-contextual setting. For instance, if we can show that having a \begin{math}\delta\end{math} unpersonalized (non-contextual policy) can incur at most \begin{math}{\cal O}(1)\end{math} regret per time step over an optimal contextual bandit. However, if we set $\delta$ to shrink at a certain rate, for instance, $\delta=1/t$ then we may still get a logarithmic regret.

We evaluate our results on several contextual bandit data-sets such as classification based public datasets
In the classification setting, we show that it is possible to achieve regret bounds equivalent to fully contextual baselines while reducing the number of contextual treatments in favor of non-contextual treatments.

\section{Preliminaries}

\subsection{Problem Formulation} \label{problem}
In the stochastic contextual bandit setting (see \cite{bubeck2012regret} for a survey), there is a finite set of arms $\cA = \{1, \dots, K\}$.
At time $t$, the environment produces $(x_t, r_t(1), \dots, r_t(K)) \sim \cD$, where $x$ is a $d$-dimensional context and $r_t(a)$ is the reward associated with each arm $a \in \cA$.

When the recommender policy selects arm $a_t$, the observables are $(x_t, a_t, r_t(a_t))$.
In particular, there is partial observability and only the reward $r_t(a_t)$ for the chosen arm $a_t$ is observed.
At each time $t$, the optimal assignment is the arm with the maximum expected reward and is denoted as \begin{math}a_t^* = \argmax_{a \in \cA} \field{E}[r_t(a) | x_t]\end{math}.

The goal of the policy is to find an assignment rule that sequentially assigns an arm to  minimize the cumulative expected regret over horizon $T$
\begin{align*}
\text{Regret}(T) = \sum_{t=1}^T \field{E} [r(a^*_t) - r(a_t)]
\end{align*}
where the assignment rule is a function of the previous observations \begin{math}\left(x_\tau, a_\tau, r_\tau(a_\tau)\right)\end{math} for \begin{math}\tau=1,\dots,t-1\end{math} and of the new context $x_t$.

\subsection{Exploration vs. Exploitation} \label{explore}
Therefore, the decision-maker has to balance exploring arms for which there is limited knowledge in order to learn and exploiting the accumulated knowledge in order to attain higher rewards.

Two established approaches for balancing the exploration vs. exploitation trade-off in stochastic contextual bandits are the linear upper confidence bound (LinUCB) algorithm \cite{li2010contextual} and the linear Thompson sampling (LinTS) algorithm \cite{agrawal2013thompson} as well as their generalized linear (particularly logistic) counterparts \cite{chapelle2011empirical,li2017provably}.
These algorithms postulate that the expected reward of arm $a$ conditional on the context $x$ can be modeled as a linear or a generalized linear function of the context with unknown parameters $w_a$.
Then, at each time $t$, they use the historical observations \begin{math}\left\{\left(x_\tau, a_\tau, r_\tau(a_\tau)\right)\right\}_{\tau=1}^{t-1}\end{math} and regularized linear or logistic regression to form an upper confidence bound or posterior (exact or approximate) on the unknown parameters \begin{math}\theta_a\end{math} of each arm \begin{math}a \in \cA\end{math}. Finally,  the upper confidence bound or the posterior is used to balance exploration vs. exploitation when deciding the arm $a_t$ for the new context $x_t$.

A simple and popular heuristic for bandit problems is the \begin{math}\epsilon\end{math}-greedy exploration strategy \cite{sutton2018reinforcement}.
According to this strategy, at every time $t$ the decision-maker computes point estimates \begin{math}\hat{w}_a\end{math} for each arm \begin{math}a \in \cA\end{math} based on the historical observations \begin{math}\left\{\left(x_\tau, a_\tau, r_\tau(a_\tau)\right)\right\}_{\tau=1}^{t-1}\end{math} and uses these point estimates to find the arm with the highest predicted expected reward for context $x_t$.
Then, the decision-maker selects the best predicted arm with probability \begin{math}1-\epsilon\end{math} and with probability \begin{math}\epsilon\end{math} selects an arm from \begin{math}\cA\end{math} uniformly at random.

Both Thompson sampling and UCB for contextual bandits have strong regret bound guarantees, however Thompson sampling tends to perform much better in practice \cite{chapelle2011empirical,russo2018tutorial,dimakopoulou2018balanced}.
On the other hand, $\epsilon$-greedy has sub-optimal guarantees compared to both Thompson sampling and UCB, but is popular in practice due to its simplicity and generally good performance.

\section{Related Work}

We focus on  algorithms for the stochastic contextual bandit problem with binary rewards, but all the presented algorithms can be extended to real number rewards.
We first present two well-known baselines from the literature; a contextual bandit algorithm that models the expected reward of each arm conditional on the context as a logistic function and a non-contextual bandit algorithm that does not take into account the context during the decision making.
Subsequently, we present our approach, the selectively contextual bandit, which interpolates between the contextual and the non-contextual bandit depending on the predicted benefit from taking the contextual information into account in each time period.
All three algorithms can be paired with any of the exploration schemes outlined in Section \ref{explore}.
Due to space limitations, we present the Thompson sampling version of the algorithms, which can be readily adapted to the UCB and $\epsilon$-greedy versions.

\subsection{$K$-Armed Bernoulli Bandit}
In the non-contextual formulation, the decision-maker does not take into account the context of every time period but rather tries to learn the unpersonalized, globally optimal arm while balancing the exploration vs. exploitation trade-off.
One straightforward approach is to model this problem as a $K$-armed Bernoulli bandit with independent arms $\cA$ \cite{thompson1933likelihood}.
In this formulation, the reward of arm $a$ follows a Bernoulli distribution with mean $\theta_a$.
It is standard to model the mean reward of arm $a$ using a Beta distribution with parameters $\alpha_a$ and $\beta_a$, since it is the conjugate distribution of the binomial distribution.
At every time $t$, the agent draws a sample mean reward $\hat{\theta}_a \sim \text{Beta}\left(\alpha_a, \beta_a\right)$ for each arm $a \in \cA$ and selects arm $a_t = \argmax_{a \in \cA} \hat{\theta}_a$.
Based on the observed reward $r_t(a_t)$, the decision-maker updates the posterior distribution on $\theta_a$.
Algorithm \ref{alg:K-armed} presents the approach.

\begin{algorithm}
\caption{Non-Contextual $K$-Armed Bernoulli Bandit}
\label{alg:K-armed}
\textbf{Require:} Initial $\alpha_a$ and $\beta_a$ for all $a \in A$ (default value: 1)
  \begin{algorithmic}[1]
    \FOR{$t = 1, \dots, T$}
      \FOR{each arm $a \in A$}
        \STATE Sample $\hat{\theta}_a \sim \text{Beta}\left(\alpha_a, \beta_a\right)$
      \ENDFOR
      \STATE Select arm $ a_t = \argmax_{a \in A} \hat{\theta}_a $
      \STATE Observe reward $r_t(a_t)$, where $r_t \sim \cD(\cdot | x_t)$
      \IF {$r_t(a_t) = 1$}
        \STATE $\alpha_{a_t} = \alpha_{a_t} + 1$
      \ELSE
        \STATE $\beta_{a_t} = \beta_{a_t} + 1$
      \ENDIF
    \ENDFOR
  \end{algorithmic}
\end{algorithm}

\subsection{Generalized Linear Bandit} \label{GLM}
Linear bandits \cite{li2010contextual,agrawal2013thompson} and generalized linear bandits (particularly logistic) \cite{chapelle2011empirical,li2017provably}  are widely used in web services for the personalization of news recommendation, advertising and search.
Generalized linear bandits (logistic regression in particular) have demonstrated stronger performance than linear bandits in many applications where rewards are binary.
In this section, we model the stochastic contextual bandit problem as a generalized linear bandit, as in \cite{chapelle2011empirical}.

The decision-maker models the expected reward of arm $a$, $\mu_a = \field{E}[r(a) | x]$, as a logistic function of context $x$  with parameters  $\theta_a \in \field{R}^d$,
$\mu_a = \field{P}(r(a) = 1| x) = \sigma(\theta_a^\top x)$
where $\sigma(z) \equiv \frac{1}{1 + \exp(-z)}$ is the sigmoid function.
The posterior distribution on the parameters $\theta_a$ of each arm $a \in \cA$ is approximated by a multivariate Gaussian distribution updated via the Laplace approximation.
Specifically, the decision-maker starts with a multivariate Gaussian prior each $\theta_a$ with mean $\boldsymbol{\mu}_0 = \textbf{0} \in \mathbb{R}^{\ell m}$ and covariance matrix $\mathbf{\Sigma}_0 = \lambda \cdot \mathbb{I}_{\ell m}$, where $\mathbb{I}_{\ell m}$ is the ${\ell m} \times {\ell m}$ identity matrix and $\lambda$ is a regularization parameter.

 \begin{algorithm}
\caption{Generalized Linear Bandit}
\label{alg:GLM}
\textbf{Require:} Parameters of weight prior $\boldsymbol{\mu}_0$ and $\boldsymbol{\Sigma}_0$
  \begin{algorithmic}[1]
  \STATE Draw weight sample $\hat{\textbf{w}} \sim \mathcal{N}(\boldsymbol{\mu}_0, \boldsymbol{\Sigma}_0)$
    \FOR{$t = 1, \dots, T$}
      \FOR{each arm $\textbf{a} \in A$}
        \STATE Compute $\hat{\theta}(\textbf{a}) = \frac{1}{1 + \exp(- \hat{\textbf{w}}^\top {\textbf{x}_{\textbf{a}}})}$
      \ENDFOR
      \STATE Select arm $\textbf{a}_t = \argmax_\textbf{a} \hat{\theta}(\textbf{a})$
      \STATE Observe reward $r_t \sim \cD(r | \textbf{a}_t)$
      \STATE Update weight posterior parameters $\boldsymbol{\mu}_t$ and $\boldsymbol{\Sigma}_t$
      \STATE Draw a new weight sample $\hat{\textbf{w}} \sim \mathcal{N}(\boldsymbol{\mu}_t, \boldsymbol{\Sigma}_t)$
    \ENDFOR
  \end{algorithmic}
\end{algorithm}

The log-posterior of $\textbf{w}$ at time $t$ is
{
\medmuskip=2mu
\thinmuskip=2mu
\thickmuskip=3mu
\begin{align*}
&\log(\mathbb{P}(\textbf{w} | \textbf{x}_{\textbf{a}_t}, r_t)) \propto
-\frac{1}{2} (\textbf{w} - \boldsymbol{\mu}_{t-1})^\top \boldsymbol{\Sigma}_{t-1}^{-1} (\textbf{w} - \boldsymbol{\mu}_{t-1}) + \\
&\qquad+r_t \log\left(  \sigma(\textbf{w}^\top {\textbf{x}_{\textbf{a}_t}}) \right) + (1- r_t) \log\left(1 -  \sigma(\textbf{w}^\top {\textbf{x}_{\textbf{a}_t}})\right)
\end{align*}}The posterior mean of $\textbf{w}$ is the maximum a posteriori estimate $\boldsymbol{\mu}_t = \textbf{w}_{\text{MAP}} = \argmax_\textbf{w} \log(\mathbb{P}(\textbf{w} | \textbf{X}, \textbf{r})) $
and the posterior covariance matrix of $\textbf{w}$ is
$\boldsymbol{\Sigma}_t^{-1} = \boldsymbol{\Sigma}_{t-1}^{-1} +  \sigma(\textbf{w}_\text{MAP}^\top {\textbf{x}_{\textbf{a}_t}}) (1 - \sigma(\textbf{w}_\text{MAP}^\top {\textbf{x}_{\textbf{t}}})) \textbf{x}_{\textbf{a}_t} \textbf{x}_{\textbf{a}_t}^\top $.
To choose the next arm, the agent draws a weight sample $\hat{\textbf{w}} \sim \mathcal{N}(\boldsymbol{\mu}_t, \boldsymbol{\Sigma}_t)$ and forms an estimate of the expected reward $\hat{\theta}(\textbf{a}) = \sigma(\hat{\textbf{w}}^\top {\textbf{x}_{\textbf{a}}})$ of each arm $\textbf{a} \in \mathcal{K}$ based on this weight sample.  Then, the agent plays arm $\textbf{a}_t = \argmax_\textbf{a} \hat{\theta}(\textbf{a})$. Algorithm \ref{alg:GLM} outlines the approach.

\section{Selectively Contextual Bandit} \label{scb}
Given the constituent policies - one contextual and the other non-contextual, we now provide the details of a hybrid policy that selectively switches between the two policies. At each time step, the two policies are used to determine their optimal arm assignments. If the arms selected from the two policies are different, rewards for the two arms using the contextual policy are estimated. The estimated rewards are then compared to determine if the predicted reward from the contextual policy for the arm selected by the contextual policy is significantly better than the arm selected by the non-contextual policy. If so, SCB selects the contextual winner, if not the non-contextual winner is used. The policies are then updated with the observed reward once the SCB makes its  final selection. The overall algorithm is sketched out in Algorithm \ref{alg:scb}.
\begin{algorithm}
\caption{Selectively contextual bandits}
\label{alg:scb}
\textbf{Require:} $\pi$ : Non-contextual policy, $\pi_c( x)$ : Contextual policy
  \begin{algorithmic}[1]
    \FOR{$t = 1, \dots, T$}
      \STATE Select arm $a_{nc} = \pi$
      \STATE Select arm $a_{c} =  \pi_c(x)$
      \STATE Predict reward $r_{nc}(a_{nc})$ and $r_{c}(a_{c})$
      \IF {$\delta$($r_{c}(a_{c})$, $r_{nc}(a_{nc})$) $>$ $\lambda$}
        \STATE $a_{scb} = a_{c}$
      \ELSE
        \STATE $a_{scb} = a_{nc}$
      \ENDIF
      \STATE Update $\pi$ and $\pi_c( x)$
    \ENDFOR
  \end{algorithmic}
\end{algorithm}

In our scheme, we evaluate two different ways of calculating the non-contextual winner: mean and beta-Bernoulli. Further, we also consider two different formulations of the delta operation in the algorithm: ratio or relative difference (please find the details in Section \ref{compared-models}). Finally, we allow for shrinking or annealing the $\delta$ threshold by a constant decay rate on a specified time schedule. This is implemented by shrinking the $\delta$ threshold by a specified constant rate at specified epochs.

\section{Experiments}
We hypothesize that a fully personalized policy should always perform better in regret analysis comparisons. But there exist real-world use cases where a hybrid policy may be desired. For example, a hybrid policy may have benefits such as avoiding filter bubbles and enabling users to participate in the zeitgeist or enabling greater overlap in shared experiences in the ever increasing personalized web. No public datasets exist to verify this claim, however. So we restrict our experiments to show that the regret of a hybrid policy is comparable to a purely contextual one. We evaluate our approach in a classification setting using public datasets, and we present and discuss our results in this section.
\subsection{Experiments on Public Datasets}
\paragraph{Multiclass Classification with Multi-Armed Bandits.} When experimenting with and comparing different contextual bandit algorithms, it is common to transform multiclass classification tasks into multi-armed bandit formulations \cite{Dudik2011}. We make the assumption that the observations are sampled from a fixed distribution and are independent and identically distributed. In both the non-contextual and contextual bandit setting, the number of classes corresponds to the number of arms. In the contextual setting, the features of each data sample correspond to that sample’s context. Accompanying each data sample is the ground truth class label. In a multiclass classification problem, the task is to learn a model that correctly assigns the correct class label to each data sample in a test set. Correspondingly, in the adaptation to a bandit problem, the goal is to learn an assignment policy that assigns the optimal (correct) arm to each sample. In the multiclass classification setting, we are attempting to learn a model that minimizes the classification error, which corresponds to the policy’s expected regret in the multi-armed bandit setting. In our implementation of the various non-contextual bandits, the assignment policy opts not to use the sample’s features (or context) during arm assignment or when updating the posterior distribution of each arm. In each non-contextual bandit’s contextual counterpart, we do leverage the additional information of the sample’s features to inform arm assignment and when updating the posterior distributions. In both the non-contextual and contextual settings, after each time time-step $t$, an arm is assigned to sample $x_t$. If the arm assignment is correct, the agent incurs a reward of one. If the arm assignment is incorrect, then the agent incurs a regret of 1. When comparing the performance of various contextual bandit algorithms in this multiclass classification problem setting, it is common to perform regret analysis and visualize the regret graphs over the history of observations using the normalized cumulative regret.
\paragraph{Experimental Set-up.} We use the Open Media Library (OpenML) \cite{OpenML2013} to collect 20 publicly available classification datasets. The datasets we use span various domains such as healthcare, biology, ecology, and computer vision and vary with regards to their attributes i.e., number of observations, classes, and features. As part of pre-processing, the categorical feature columns are one-hot encoded. Before each run, we randomly shuffle the dataset. We run our suite of bandit algorithms on each of the 20 datasets for different model hyperparameters. For the SCB bandits, we vary the various input parameters including the delta threshold value, the delta shrinkage rate, and delta shrinkage schedule. These SCB input parameters control the amount of non-contextual decisions that are made in favor of contextual ones, with the option to anneal $\delta$ by a constant rate at various time-steps in the horizon.

\paragraph{SCB Input Parameter Selection.} In this paragraph we offer a more in-depth explanation on how we pick the initial delta rates as well as how we selected the subsequent annealing rate and annealing schedule. But first, we provide some intuition for why delta rate annealing may be desired or even necessary in some cases. In the earlier timesteps of a contextual model that is learning the optimal arm assignment policy for a particular dataset, the model has not yet learned a good policy because it has not seen enough samples. Thus, we choose a higher $\delta$ value in this lower sample regime, making noncontextual decisions more frequently. As time goes on, the contextual bandit begins to learn a better arm assignment policy so we shrink the rate accordingly to account for this higher degree of confidence. During our evaluation of our SCB models on the OpenML datasets, we began by selecting an initial delta rate of 1.0 for SCB models using ratios for expected reward comparisons and 0.0 for SCB models using relative differences for expected reward comparisons. This was for sanity checking that SCB policies with thresholds meant to select the contextual winner every time matched the fully contextual baseline policy. We then chose reasonably high thresholds, e.g. 1.5 for SCB policies based on ratio comparisons and 0.5 for those based on relative difference comparisons, without an annealing schedule to observe the commutative regret over the horizon using a policy that selects a large number of noncontextual decisions in favor of contextual ones. As expected, the fully contextual bandit always performed better than our SCB models under these high initial delta rates. We then steadily decreased this initial decay rate until finding a starting rate that allowed our policy to roughly match the regret bounds of the fully contextual policy. Having found this initial delta rate, we then applied a shrinkage scheduler that was not aggressive, for doing so also decreases the number of noncontextual decisions that are made. Future work includes automatically finding the SCB parameters that allow the maximal noncontextual decisions to be made while staying on par or improving upon the fully contextual policy.

\subsubsection{Compared Models}\label{compared-models}
The following lists the multi-armed bandit algorithms we evaluated for performance comparison. It includes different flavors of SCB bandits and baseline models. If the model name ends in ``Ratio,'' the ratio of the contextual to noncontextual expected reward is compared against the SCB delta threshold at each time-step $t$ to decide between differing arm assignments. If the model name ends in ``Diff,''  then the relative difference of the contextual and noncontextual expected reward is compared against the SCB delta threshold. $\epsilon=.2$ for all $\epsilon$-greedy bandits evaluated.
\begin{itemize}
\item \textbf{IndependentBernoulliArmsEGAgent}\cite{thompson1933likelihood} non-contextual beta-Bernoulli bandit using $\epsilon$-greedy as the explore/exploit strategy.
\item \textbf{LogisticRegressionEGAgent}\cite{chapelle2011empirical,li2017provably} contextual bandit that models the expected reward of each arm as a logistic function using $\epsilon$-greedy as the explore/exploit strategy.
\item \textbf{SCBEGAgent\_Ratio} SCB agent that interpolates between treatment decisions made by IndependentBernoulliArmsEGAgent and LogisticRegressionEGAgent, using ratio of expected rewards for comparisons against SCB $\delta$ parameter.
\item \textbf{meanSCBEGAgent\_Ratio} SCB agent with LogisticRegressionEGAgent as its base model, using ratio of expected rewards for comparisons against SCB $\delta$ parameter. The noncontextual winning arm is determined to be the arm with the maximum average expected reward taken across all contexts in the history.
\item \textbf{SCBEGAgent\_Diff} SCB agent that interpolates between treatment decisions made by IndependentBernoulliArmsEGAgent and LogisticRegressionEGAgent, using the relative difference of expected rewards for comparison against SCB $\delta$ parameter.
\item \textbf{meanSCBEGAgent\_Diff} SCB agent with LogisticRegressionEGAgent as its base model, using relative differences during comparisons against SCB $\delta$ parameter. The noncontextual winning arm is determined to be the arm with the maximum average expected reward taken across all contexts in the history.
\item \textbf{IndependentBernoulliArmsTSAgent} non-contextual beta-Bernoulli bandit using Thompson Sampling as explore/exploit strategy.
\item \textbf{LogisticRegressionTSAgent}\cite{agrawal2013thompson} contextual bandit that models the expected reward of each arm as a logistic function using Thompson Sampling as the explore/exploit strategy.
\item \textbf{SCBTSAgent\_Ratio} SCB agent that interpolates between treatment decisions made by IndependentBernoulliArmsTSAgent and LogisticRegressionTSAgent, using ratio of expected rewards for comparisons against SCB $\delta$ parameter.
\item \textbf{meanSCBTSAgent\_Ratio} SCB agent with LogisticRegressionTSAgent as its base model, using ratio of expected rewards for comparisons against SCB $\delta$ parameter. The noncontextual winning arm is determined to be the arm with the maximum average expected reward taken across all contexts in the history.
\item \textbf{SCBTSAgent\_Diff} SCB agent that interpolates between treatment decisions made by IndependentBernoulliArmsTSAgent and LogisticRegressionTSAgent, using the relative difference of expected rewards for comparison against SCB $\delta$ parameter.
\item \textbf{meanSCBTSAgent\_Diff} SCB agent with LogisticRegressionTSAgent as its base model, using the relative difference of expected rewards for comparison against SCB $\delta$ parameter. The noncontextual winning arm is determined to be the arm with the maximum average expected reward taken across all contexts in the history.
\item \textbf{IndependentBernoulliArmsUCBAgent}\cite{li2010contextual} non-contextual beta-Bernoulli bandit using UCB as explore/exploit strategy.
\item \textbf{LogisticRegressionUCBAgent} contextual bandit that models the expected reward of each arm as a logistic function using UCB as the explore/exploit strategy.
\item \textbf{SCBUCBAgent} SCB agent that interpolates between treatment decisions made by IndependentBernoulliArmsUCBAgent and LogisticRegressionUCBAgent, using the relative difference of expected rewards for comparison against SCB $\delta$ parameter.
\item \textbf{meanSCBUCBAgent} SCB agent with LogisticRegressionUCBAgent as its base model, using the relative difference of expected rewards for comparison against SCB $\delta$ parameter. The noncontextual winning arm is determined to be the arm with the maximum average expected reward taken across all contexts in the history.
\end{itemize}

\subsubsection{Results}
\begin{figure}[ht!]
    \begin{subfigure}{0.4\textwidth}
    \includegraphics[width=\textwidth]{fig_679-TS.png}
    \caption{}
    \label{fig:679-ts}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.4\textwidth}
    \includegraphics[width=\textwidth]{fig_679-UCB.png}
    \caption{}
    \label{fig:679-ucb}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.4\textwidth}
    \includegraphics[width=\textwidth]{fig_1413-TS.png}
    \caption{}
    \label{fig:1413-ts}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.4\textwidth}
    \includegraphics[width=\textwidth]{fig_1508-UCB.png}
    \caption{}
    \label{fig:1508-ucb}
    \end{subfigure}
    \caption{Regret analysis comparison between SCB and baseline algorithms on OpenML public multiclass classification datasets.}
    \label{fig:openml}
\end{figure}
Figure \ref{fig:openml} shows a regret analysis comparison between the various SCB and baseline algorithms described in Section \ref{compared-models} on three different OpenML public multiclass classification datasets. Due to space constraints, we have not included the full set of graphs for each of the 20 datasets on each of the 16 models. The $x$ coordinate is the timestep and the $y$ coordinate is the normalized cumulative regret $\frac{1}{n}\sum_{t=1}^{n} (1-r_t(a_t))$. $t$ is the timestep, the arm selected by the bandit at time $t$ is $a_t \in \{1,2,...K\}$, where $K$ is the number of arms or classes in the dataset with $n$ observations and with reward function $r_t(a_t) = 1 \{a_t = c_t\}$ where $c_t$ is the true class of context $x_t$. The lower the cumulative regret the better. We run each agent 20 times over each dataset (the dataset is reshuffled at the beginning of each run). We set the horizon to 3000 observations, updating the prior distributions and the logistic models after every batch of 100 timesteps.

Figure \ref{fig:679-ts} shows a comparison of the various Thompson Sampling bandits we evaluated on OpenML dataset id 679, a dataset of sleep state measurements. We observe that all five bandits display the same regret curves. However, while the baseline model, LogisticRegressionTSAgent, made contextual decisions at every timestep $t$, the SCBTSAgent\_Ratio bandit chose noncontextual decisions over contextual ones 7.2\% of the time averaged across the 20 runs. Similarly, the meanSCBTSAgent\_Ratio bandit chose noncontextual decisions over contextual ones 10.4\% of the time, SCBTSAgent\_Diff 12.1\% of the time, and meanSCBTSAgent\_Diff 14.2\% of the time. Figure \ref{fig:679-ucb} shows a comparison of the various UCB bandits we evaluated on the same dataset. Again, we observe that all three bandits display the same regret curves. The baseline model, LogisticRegression UCBAgent, made contextual decisions at every timestep, SCBUCBAgent chose the noncontextual decision over the contextual decision a mean percentage of 33.4\% of the times, and meanSCBUCBAgent 14.7\% of the times.

Figure \ref{fig:1413-ts} shows a comparison of the various Thompson Sampling bandits we evaluated on OpenML dataset id 1413, the Iris dataset. We observe that the four SCB bandits outperform the baseline LogisticRegressionTSAgent. However, it is interesting to note that the number of noncontextual decisions made in favor of contextual ones was minimal. The SCBTSAgent\_Ratio bandit chose noncontextual decisions over contextual ones only .43\% of the time averaged across the 20 runs. Similarly, the meanSCBTSAgent\_Ratio bandit chose noncontextual decisions over contextual ones .43\% of the time, SCBTSAgent\_Diff made a few more noncontextual decisions at 3.9\%, and meanSCBTSAgent\_Diff  at 1.4\%. Figure \ref{fig:1508-ucb} shows a comparison of the various UCB bandits we evaluated on the OpenML dataset id 1508, a user knowledge dataset. Here we observe again all three models performing on par with each other with the SCBUCBAgent choosing noncontextual decision over the contextual decision a mean percentage of 9.6\% and meanSCBUCBAgent 12.1\%.

The results from the experiments run on publicly available multiclass classification datasets showed us that in many cases, there exists a hybrid policy that reduces dependence on context whilst achieving regret bounds that are on par with fully contextual baseline algorithms. In a few cases, we found a policy that made it possible to outperform a fully contextual policy. Likewise, we also observed a few cases where we were unable to find a delta value that reduced the number of contextual decisions without strictly hurting performance. Overall, we believe that SCB is an algorithm that practitioners can employ if they wish to find policies that reduce the number of the contextual decisions (or conversely, increase the number of common treatments that are assigned) without significantly impacting performance.

\iffalse
\subsection{Experiments on proprietary Dataset}
We evaluate SCB on a proprietary dataset used for item selection personalization. This is a common paradigm used in online web services that must decide which item from a set of items to impress on a user $u$ with context $x \in \mathcal{X}$. Exact details, graphs, and values are omitted to protect business-sensitive intellectual property.
\paragraph{Evaluation Metrics.} We define the Click through rate (CTR) of an item as the fraction of users who engaged with an item after being impressed with it. A controlled randomized uniform exploration of the candidate items is done by a logging policy which provides us with a dataset to evaluate the proposed SCB algorithm.  We compute off-policy replay metrics by following the method described in \cite{Li2011} to evaluate and compare the unbiased offline performance of the various online learning policies we experimented with. This method allows us to answer counterfactual questions based on the logged exploration data. In other words, we can compare offline what would have happened in historical sessions under different scenarios if the recommender system used different algorithms in an unbiased way.
\paragraph{Experimental Set-up.} We compared SCB against a multi-armed contextual bandit policy and a noncontextual multi-arm bandit policy. The contextual information of a user is represented as a feature vector provided as input to the model for predicting the probability of reward for each item $i$. Features primarily encapsulate the user’s past engagement behavior. We tested one class of SCB models, based on the same underlying logistic regression model used in production and using the ratio of expected rewards for comparisons against the SCB $\delta$ parameter. If the ratio of the expected reward of the contextual selected arm (or personalized winning item) to the expected reward of the noncontextual selected arm (or unpersonalized winning item) is not above some threshold $\delta$, then we show the unpersonalized winning selection. We calculate the policy level CTR and repeat the experiment for various SCB $\delta$ threshold values. As we sweep the delta from 1 to 10, we move from a fully personalized experience to a fully unpersonalized one. We do not anneal the delta rate over the history of contexts, i.e. we keep the delta rate constant. We ran experiments on different asset types and across data streams from different months and days.

\subsubsection{Compared Policies} \label{compared-policies}
We considered three different policies for selecting the noncontextual winning item.
\begin{itemize}
\item \textbf{SCB Global Majority Vote} The noncontextual selected item is calculated as the global majority vote item across all contexts. Within the context of user $u$, if the ratio between the expected reward of the contextual selected item to the expected reward of the noncontextual selected item is not greater than SCB input parameter $\delta$, then show user $u$ the global majority vote selection. Otherwise, show the member the personalized winning selection.
\item \textbf{SCB Country-level Majority Vote} The noncontextual selected item is calculated as the country-level majority vote item across all contexts with country location $c$. Within the context of user $u$ with country location $c$, if the ratio between the expected reward of the contextual selected item to the expected reward of the noncontextual selected item is not greater than SCB input parameter $\delta$, then show user $u$ the country-level majority vote selection. Otherwise, show the member the personalized winning selection.
\item \textbf{SCB Manually Curated Vote} The noncontextual selected item is set to an item manually pre-selected by a group of experts. Within the context of user $u$, if the ratio between the expected reward of the contextual selected item to the expected reward of the noncontextual selected item is not greater than SCB input parameter $\delta$, then show user $u$ the manually curated item. Otherwise, show the member the personalized winning selection.
\end{itemize}

\subsubsection{Results}
Results on this proprietary personalization dataset demonstrate that indeed we are able to increase the number of users who are impressed with the noncontextual decision without a significant negative impact on the overall CTR. We are able to show 20\% of the users the noncontextual decision, i.e. unpersonalized selection, instead of the contextual decision, i.e. personalized selection, while at the same time achieving a CTR that is on par with the production policy CTR that always opts to go with the contextual decision. We also observe that at a threshold value that converts 10\% of the contextual decisions to noncontextual ones, we observe an SCB policy that slightly outperforms the baseline CTR (the fully contextual policy used in production). As a we increased the SCB threshold value $\delta$, the policy CTR stayed relatively flat before sharply decreasing. It is worth noting that a policy that shows all users the noncontextual item does worse than a fully personalized experience. Intuitively, this aligns with the original intent of personalization. We conducted the same analysis for different asset types and across data streams from five different days and saw similar trends. Overall, our offline empirical results on this proprietary dataset suggest that there exists an SCB policy that reduces dependence on context that can achieve replay CTR that are on par with or even outperform those achieved by a fully contextual policy.
\fi

\subsection{Experiments on proprietary Dataset}
In an online video subscription setting, promotional title artwork has undergone a paradigm shift. The burden has shifted from attempting to appeal to as many people as possible with two or three posters to attempting to appeal to a single user in a single online video viewing session. The goal is no longer solely about capturing public joy but about capturing individual attention. Acknowledging that any given title may cover a range of thematic themes with an ensemble of cast members that may each appeal to a particular user in a different way, personalized artwork selection allows online video services to hone in on the varying taste profiles of their member-base. Artwork personalization via online learning with explore exploit and contextual multi-armed bandits is important to the member experience in a video subscription service with thousands of titles and shows available to watch.  The image personalization system algorithmically selects 1 of $N$ possible images from the title image suite to display to a user at a specific point in time or viewing session. The goal is to emphasize different themes through various artwork according to some context (user viewing history, country) in order to capture individual preferences for cast members, genres, artistic themes, etc.
\paragraph{Evaluation Metrics.} We define the Click through rate (CTR) of an item as the fraction of users who engaged with an item after being presented with it. A controlled randomized uniform exploration of the candidate items is done by a logging policy which provides us with a dataset to evaluate the proposed SCB algorithm.  We compute off-policy replay metrics by following the method described in \cite{Li2011} to evaluate and compare the unbiased offline performance of the various online learning policies we experimented with. This method allows us to answer counterfactual questions based on the logged exploration data. In other words, we can compare offline what would have happened in historical sessions under different scenarios if the recommender system used different algorithms in an unbiased way.
\paragraph{Experimental Set-up.} We compared SCB against a multi-armed contextual bandit policy and a non contextual multi-arm bandit policy. The contextual information of a user is represented as a feature vector provided as input to the model for predicting the probability of reward for each item $i$. Features primarily encapsulate the user’s past engagement behavior. We tested one class of SCB models, based on the same underlying logistic regression model used in production and using the ratio of expected rewards for comparisons against the SCB $\delta$ parameter. If the ratio of the expected reward of the contextual selected arm (or personalized winning image) to the expected reward of the noncontextual selected arm (or unpersonalized winning image) is not above some threshold $\delta$, then we show the unpersonalized winning image. We calculate the policy level take-rate and repeat the experiment for various SCB $\delta$ threshold values. As we sweep the delta from 1 to 10, we move from a fully personalized image selection experience to a fully unpersonalized one, with all members being impressed with the same images for the same titles. We do not anneal the delta rate over the history of contexts, i.e. we keep the delta rate constant. We ran experiments on different asset types and across data streams from different months and days.

\subsubsection{Compared Policies} \label{compared-policies}
We considered three different policies for selecting the noncontextual winning image.
\begin{itemize}
\item \textbf{SCB Global Majority Vote} The noncontextual selected image for title $t$ is calculated as the global majority vote image for $t$ across all contexts. Within the context of user $u$ and title $t$, if the ratio between the expected reward of the contextual selected image to the expected reward of the noncontextual selected image is not greater than SCB input parameter $\delta$, then show user $u$ the global majority vote image for $t$. Otherwise, show the member the personalized winning image.
\item \textbf{SCB Country-level Majority Vote} The noncontextual selected image for title $t$ is calculated as the country-level majority vote image for $t$ across all contexts with country location $c$. Within the context of member $m$ with country location $c$ and title $t$, if the ratio between the expected reward of the contextual selected image to the expected reward of the noncontextual selected image is not greater than SCB input parameter $\delta$, then show user $u$ the country-level majority vote image for $t$. Otherwise, show the member the personalized winning image.
\item \textbf{SCB Marketing Default} The noncontextual selected image for title $t$ is set to the marketing default image. Within the context of user $u$ and title $t$, if the ratio between the expected reward of the contextual selected image to the expected reward of the noncontextual selected image is not greater than SCB input parameter $\delta$, then show user $u$ the marketing default image for $t$. Otherwise, show the member the personalized winning image.
\end{itemize}

\subsubsection{Results}
\begin{figure}[t]
    \includegraphics[width=0.5\textwidth]{fig_industry_results}
    \caption{Comparison of three different policies for selecting the noncontextual arm in an industry image personalization system. A policy that fallbacks on the default marketing image during non-contextual SCB decisions performs best until a certain threshold.}
    \label{fig:industry}
\end{figure}
Figure \ref{fig:industry} shows a comparison of the three different policies tested for selecting the noncontextual arm (unpersonalized winning image) on the large-scale proprietary image personalization dataset. Exact details and values are omitted to protect business-sensitive intellectual property. The $x$ coordinate is the SCB threshold value. As we move from left to right, increasing the threshold, we go from a fully personalized experience (threshold of 1) to a fully unpersonalized experienced. That is, we move from a policy that selects the contextual decision every time to one that selects the noncontextual decision every time. The $y$ coordinate is the offline policy take-rate. The higher the value the better. Each of the three lines corresponds to each of the three policies described in Section \ref{compared-policies}.

The first important observation is that with the SCB Marketing Default policy in particular we are able to increase the number of users who are impressed with the noncontextual decision (in this case, the default image) without a significant negative impact on the overall take-rate. As the figure shows, we are able to show 20\% of the users the noncontextual decision, i.e. default image, instead of the contextual decision, i.e. personalized image, while at the same time achieving a take-rate that is on par with the production policy take-rate that always opts to go with the contextual decision. The second important observation is that at a threshold value that converts 10\% of the contextual decisions to noncontextual ones, we observe an SCB Marketing Default policy take-rate that slightly outperforms the baseline take-rate (the fully contextual policy used in production). In this particular setting, we saw SCB perform best when using the marketing default image as the noncontextual fallback decision. Because as a we increased the SCB threshold value $\delta$, the policy take-rate stayed relatively flat before sharply decreasing. It is worth noting that a policy that shows all users the default image does worse than a fully personalized experience and does worse than a policy that shows all users the global majority vote or country-level majority vote image. Intuitively, this aligns with original motivation to implement image personalization. We conducted the same analysis for different image asset types and across data streams from five different days and saw similar trends. Overall, our offline empirical results on this industry dataset suggest that there exists an SCB policy that reduces dependence on context that can achieve replay take-rates that are on par with or even outperform those achieved by a fully contextual policy.

\section{Conclusion}
In this paper we propose a new family of multi-armed contextual bandits called selectively contextual bandits (SCB) that selectively interpolate between contextual and non-contextual treatment decisions. Using publicly available datasets corresponding to 20 different classification tasks, we have empirically demonstrated that it is possible to increase the number of non-contextual decisions from the policy while achieving similar regret metrics as a fully-contextual policy. In fact, we observe that in some cases, it is possible to even slightly outperform a fully-contextual policy. Further, we demonstrate that SCB is flexible, accommodating different explore/exploit algorithms and allowing the ability to control the amount of contextual decisions that are made using a scheduler to anneal the SCB threshold over the history of time-steps. We hypothesize that an SCB policy is beneficial in creating rich personalized treatments while also increasing the number of shared experiences across users, potentially leading to social participation through network effects.

\section*{Acknowledgments}
Thank you to Arvind Narayanan, Jeremy Smith, Fernando Amat Gil, Hakan Ceylan, Hossein Taghavi, Christine Doig-Cardet, Sonali Sharma for enabling this body of research and for helpful discussions. This research was supported by the National Defense Science and Engineering Graduate Fellowship and in part by the National Science Foundation under Award CHS-1704444.

\bibliographystyle{named}


\section{Introduction} \label{sec:intro}

The ultra-compact family of low mass X-ray binaries (LMXBs) is composed of those systems in which a compact object, a neutron star or a black hole, accretes material from an evolved, hydrogen-deficient companion star in a tight orbit with a period $P_\mathrm{orb}$~$<80$~min \citep{Rappaport1982,Verbunt1995}. Three evolutionary channels, depending on the nature of the donor star, have been proposed to explain the origin of such compact systems: the white dwarf channel, the helium star channel and the evolved main-sequence star channel \citep[see e.g.,][]{Nelemans2010b, Heinke2013}. Distinguishing between these formation paths is not always straightforward, since the final state of the donor is similar for the three scenarios. In this regard, the study of the progenitors of ultra-compact systems (i.e., before the orbital period becomes shorter than 80~min) may be key to shed light on the origin of this family.

The neutron star LMXB 4U\,1812--12\ is a strong ultra-compact candidate. Its persistently low X-ray luminosity ($\sim4\times10^{35}$~$\mathrm{erg~s}^{-1}$) and low optical--to--X-ray flux ratio suggest that it harbors a small accretion disk \citep{Paradijs1994, Bassa2006, IntZand2007}. In addition, its optical spectrum lacks hydrogen spectral features, which suggest a hydrogen-exhausted donor star \citep{ArmasPadilla2020}.

In order to confirm the ultra-compact nature of 4U\,1812--12, we performed a detailed time-resolved photometric study using data taken with the HiPERCAM imager on the 10.4-m Gran Telescopio Canarias (GTC).

\section{Observations and reduction}\label{sec:Obs}

We obtained images of 4U\,1812--12\ with
HiPERCAM \citep{dhillonetal21-1} on the GTC in La Palma. This high-speed camera uses four dichroic beamsplitters and five frame-transfer CCDs to simultaneously image the $u_\mathrm{s}$, $g_\mathrm{s}$, $r_\mathrm{s}$, $i_\mathrm{s}$ and $z_\mathrm{s}$ optical bands\footnote{HiPERCAM is equipped with ''Super'' SDSS filters, high-throughput versions of the SDSS filters, hence the "s" subscript.}. The CCD detectors were used in full-frame mode with slow readout and no binning. The observations presented here were taken on 2021 June 15 with an exposure time of 12.9\,s (with only 8\,ms dead time between exposures) for 1129 frames ($\sim 4$-hour total coverage). The cadence in the $u_\mathrm{s}$ and $g_\mathrm{s}$ bands was slower by a factor of 15 in an attempt to increase the signal-to-noise ratio in these bands. The data were reduced using the HiPERCAM pipeline\footnote{\url{https://github.com/HiPERCAM/}}. We block averaged the $r_\mathrm{s}$, $i_\mathrm{s}$ and $z_\mathrm{s}$ images and obtained 60 images per band with improved signal-to-noise ratio. We then extracted the count rates of 4U\,1812--12\ and a comparison star\footnote{\href{http://vizier.u-strasbg.fr/viz-bin/VizieR-S?Gaia\%20DR2\%204153779729434598528}{Gaia\,DR2\,4153779729434598528}} via variable aperture photometry by direct summing of the sky subtracted flux over the aperture, i.e. with no profile weighting. The signal-to-noise ratios of the $u_\mathrm{s}$ and $g_\mathrm{s}$ images were insufficient to extract reliable fluxes. In fact, the source is not detected in either the $u_\mathrm{s}$ or $g_\mathrm{s}$ stacked images, so these bands are not discussed further in this paper. The measured seeing gradually degraded during the last third of the observation from a median value in the $z_\mathrm{s}$ band of 1.1 to 1.8 arcsec.

\begin{figure}
\centering
\includegraphics[width=0.9\columnwidth]{LC_mag_3pannel.pdf}
\caption{GTC/HiPERCAM $r_\mathrm{s}$$i_\mathrm{s}$$z_\mathrm{s}$ light curves of 4U\,1812--12. The individual images were block averaged into 60 bins prior to flux extraction to enhance the signal-to-noise ratio.}
\label{fig:riz_lc}
\end{figure}

\section{Period analysis and results}\label{sec:Res}

The light curves of 4U\,1812--12\ are shown in Fig.~\ref{fig:riz_lc}. The average magnitudes of the system are $r_\mathrm{s} = 23.17 \pm 0.02$, $i_\mathrm{s} = 21.93 \pm 0.01$ and $z_\mathrm{s} = 21.28 \pm 0.01$\,mag. To compute the light curves we used the Pan-STARRS Data Release 1 magnitudes of the comparison star $r = 20.33 \pm 0.02$, $i = 18.58 \pm 0.01$ and $z = 17.42 \pm 0.01$\,mag.

We analyzed the $z_\mathrm{s}$-band light curve of 4U\,1812--12\ using the analysis of variance (ANOVA) method \citep{schwarzenberg-czerny96-1}. This uses the ANOVA statistic, $\Theta$, to assess the goodness of fits to the data with periodic orthogonal polynomials. The periodogram is displayed in Fig.~\ref{fig:z_pgram} and favors a period of $114.5 \pm 3.5$~min ($0.0795 \pm 0.0024$~d), where the 1-$\sigma$ confidence interval was calculated using a postmortem analysis \citep{schwarzenberg-czerny91-1} and is defined as the width of the periodogram peak at the mean noise power level (MNPL; $\Theta_\mathrm{MNPL} = 2.52$ in this case) in its vicinity. We also used the ANOVA method with the $r_\mathrm{s}$ and $i_\mathrm{s}$ light curves and obtained consistent results. In Fig.~\ref{fig:riz_lc_folded} we show the average-subtracted light curves phase binned on this period. A peak-to-peak amplitude of $\approx 0.4$~mag is found for the three bands.

\begin{figure}
\includegraphics[width=1.0\columnwidth]{hcam_4U1812_SDSSz_ANOVA_pgram_PY.pdf}
\caption{Analysis of variance (ANOVA) periodogram of the $z_\mathrm{s}$-band light curve of 4U\,1812--12. A period of $114.5 \pm 3.5$~min ($0.0795 \pm 0.0024$~d) is favored.}
\label{fig:z_pgram}
\end{figure}

The observed photometric modulation can be a reflection of the orbital period and may be produced by either X-ray irradiation of the donor star or a superhump with a period a few per cent longer than the orbital period, caused by the precession of an eccentric accretion disk \citep{VanParadijs1988, Whitehurst1991}. While X-ray heating of the donor produces sinusoidal light curves   \citep[e.g.,][]{Paradijs1995}, changes in the disk size and shape and resonance between the Keplerian orbits in the disk and the orbital motion of the donor star can produce a more complex morphology in the superhump modulations \citep{Odonoghue1996, Haswell2001, Zurita2008}.

The sawtooth-like modulation that we detected (Fig.~\ref{fig:riz_lc}) supports the superhump scenario, and the phase-binned light curves presented in Fig.~\ref{fig:riz_lc_folded} are similar to the superhumps observed in cataclysmic variables (CVs; \citealt{Patterson2005}) and X-ray binaries \citep{Odonoghue1996,Zurita2002,Zurita2008}. Furthermore, the modulation is found to be color independent, as is also the case for superhumps (\citealt{Zurita2008} and references therein). Persistent CVs show permanent superhumps with typical amplitudes (peak-to-peak) of $\approx0.1$~mag \citep{Smak2010}. These are smaller than the $\approx 0.4$~mag amplitude in 4U\,1812--12\ (Fig. \ref{fig:riz_lc_folded}). However, superhumps with larger amplitudes ($\approx 0.25-0.6$) have been observed during CV superoutbursts \citep{Smak2010}. These are also consistent with those found in superhumps detected during LMXB outbursts ($\approx 0.1$ to 0.6~mag; \citealt{Odonoghue1996, Zurita2008,Thomas2022a}) and permanent superhumps in persistent LMXBs. For instance, a $\approx 0.6$~mag amplitude permanent superhump has been detected in the ultra-compact X-ray binary 4U~1915$-$05 \citep{Callanan1995, Chou2001, Haswell2001, Retter2002}, which has a low persistent luminosity, similar to 4U\,1812--12.

\begin{figure}
\centering
\includegraphics[width=0.9\columnwidth]{LC_DELTAmag_phase_3pannel_.pdf}

\caption{GTC/HiPERCAM $r_\mathrm{s}$$i_\mathrm{s}$$z_\mathrm{s}$ light curves phase binned (20 bins) on the $114.5 \pm 3.5$~min period, in which the respective average magnitudes have been subtracted. The phases are computed relative to the time of the first data point. Colors are the same as in Fig.~\ref{fig:riz_lc}. The whole cycle has been plotted twice for continuity.}
\label{fig:riz_lc_folded}
\end{figure}

It is important to bear in mind, however, that our observation spans two and a half periods, and therefore we can not fully discard a non-coherent origin for the modulation, although we deem it unlikely. Rapid aperiodic variability with a suggested origin in the accretion disk is commonly observed in X-ray binaries \citep{Shahbaz2003, Zurita2003, Hynes2004b, Shahbaz2013, Casares2014}. However, this flickering activity results in erratic variations of typically very short time scales (from seconds to minutes). Longer flares with time scales of hours have also been observed, particularly in systems with long orbital periods (e.g., the 6-h flares in the 6.5-d orbital period LMXB V404~Cyg, \citealt{Casares2014}). However, such a long orbital period would require a much higher mass-accretion rate in 4U\,1812--12\ to sustain its persistent nature, which would not be consistent with its observed X-ray luminosity. Flaring events produced by the compact jet are also unlikely, since the jet is not expected to make a dominant contribution to the optical emission in neutron star X-ray binaries \citep[e.g.,][]{Russell2006,Migliari2010}. Further, they should be more prominent in the red bands \citep[e.g.,][]{Gandhi2016}. All considered, although we cannot fully discard that the modulation may result from the detection of two $\approx 114$~min long, color-independent and remarkably similar (in shape and amplitude) flares, a superhump origin for the observed modulation seems to be the most plausible scenario. We therefore tentatively propose an orbital period $P_\mathrm{orb}$\ $\approx 114$~min for 4U\,1812--12.\footnote{Negative superhumps, with a period slightly shorter than the $P_{\rm orb}$, have also been detected in some systems, and are suggested to be produced by retrograde precession of a tilted disk.}

\section{Discussion}\label{sec:Disc}

4U\,1812--12\ is a strong ultra-compact X-ray binary candidate. It shows several of the distinctive characteristics of the class, related to their compact geometry (small accretion disk) and nature (hydrogen-exhausted donor; see more details in \citealt{IntZand2007}; \citealt{ArmasPadilla2019b}). In particular, since its discovery 50 years ago by the Uhuru mission \citep{Forman1976}, the system has been repeatedly detected in the X-rays at $\sim$~(3--10)~$\times10^{-10}~\mathrm{erg~cm}^{-2}~\mathrm{s}^{-1}$ (depending on the X-ray band), which together with the suggested distance of $2.3-4.6$~kpc  \citep{Cocchi2000, Jonker2004,Galloway2020}, translates to a persistent X-ray luminosity of $\sim$~0.1--1~$\times 10^{36}~\mathrm{erg~s}^{-1}$ \citep[e.g.][]{Warwick1981, Barret2003, Wilson2003, Muno2005b, Tarana2006}. This persistent activity at such low X-ray luminosity prompted \citet{IntZand2007} to propose an ultra-compact orbit for the source, since only small disks can be entirely ionized at such low accretion rates \citep{Lasota2001}. In the same way, \citet{Bassa2006} suggested a short orbital period based on the very dim optical counterpart ($g \simeq 25$, $r \simeq 23$\,mag; \citealt{ArmasPadilla2020}), since the X-ray (to optical) reprocessing scales with the size of the accretion disk \citep{Paradijs1994}. Yet, possibly the most compelling evidence for the ultra-compact nature of 4U\,1812--12\ is the absence of hydrogen features in its optical spectrum \citep{ArmasPadilla2020}, in particular H$\alpha$, which is typically the most prominent optical emission line in LMXBs with hydrogen-rich donor stars \citep{Charles2006}.

\begin{figure}$\phantom{!t}$
\centering
\begin{tabular}{c}
\includegraphics[keepaspectratio,width=\columnwidth, trim=1.0cm 0.0cm 0.0cm 1.0cm]{DIM_UCXB_v.pdf}

\end{tabular}
\caption{Stability limits for non-irradiated pure-helium disks (red solid line), for irradiated pure-helium disks (red dashed line), for non-irradiated C/O disks (blue solid line), for irradiated mixed-composition disks (green dashed line), for non-irradiated solar-composition disks (magenta solid line) and for irradiated solar-composition disks (magenta dashed line) according to \citet{Menou2002} and \citet{Lasota2008}. The black dot corresponds to 4U\,1812--12\ assuming the tentative $\approx 114$-min orbital period and the mass transfer rate reported in this work (see Section~\ref{sec:Disc}).}
\label{fig:DIM}
\end{figure}

If we assume that 4U\,1812--12\ is indeed an ultra-compact system (i.e., hydrogen poor), the numerous short thermonuclear bursts displayed by the source would need to be fuelled by pure He, pointing to a He-rich donor (see \citealt{ArmasPadilla2020}). Considering evolutionary tracks for donor stars in ultra-compact binaries, the derived mass transfer rate\footnote{We derived the mass transfer rate following \citet{Coriat2012}, assuming an average 2--10 keV X-ray unabsorbed continuum flux of $3.8\times10^{-10}~\mathrm{erg~cm}^{-2}~\mathrm{s}^{-1}$ \citep{Cocchi2000}, a bolometric correction of 2.9 \citep[for which we account a 25 per cent uncertainty;][]{IntZand2007} and a distance of $2.3-4.6$~kpc \citep{Jonker2004, Galloway2020}.} of $\sim (3.4 \pm 2.5)\times10^{-10}\,\mathrm{M}_{\odot}$~yr$^{-1}$ translates into an orbital period of $\simeq 20$~min and $\mathrel{\hbox{\rlap{\hbox{\lower4pt\hbox{$\sim$}}}\hbox{$>$}}} 35$~min for a He white dwarf and a He-star donor, respectively. However, according to disk instability models for irradiated He accretion disks, the orbital period should be $\lesssim25$~min in order to sustain a stable disk ($\lesssim40$~min in the case of a C/O disk; \citealt{Menou2002, Lasota2008}).

\subsection{An ultra-compact X-ray binary progenitor?}\label{subsec:evolved}
Our tentative $P_\mathrm{orb}$\ $\approx 114$~min lies outside the ultra-compact zone of $P_\mathrm{orb}$~$<80$~min, contrary to what was expected from the aforementioned observational properties.
The binary orbital separation is still large enough to fit a main sequence star. In fact, a 114-min orbit would require a very late M-type main sequence star or a brown dwarf to fill its Roche lobe \citep{Faulkner1972, Cox2000, Rappaport2021}. In such cases, the system mass ratio would be $q=M_{2}/M_{1} < 0.04$ (assuming  $M_{1}=1.4\,\mathrm{M}_{\odot}$), which is in agreement with the low mass ratio required to produce superhump modulations \citep{Whitehurst1991}. Further,  according to evolutionary sequences for LMXBs, systems with an initial companion star mass $\approx 0.6-3\,\mathrm{M}_{\odot}$ and an orbital period below the bifurcation period evolve by shrinking the orbit to a minimum period that can be as low as $P_{\rm orb}$\ $\simeq 80$~min \citep[see e.g.,][]{Rappaport1982, Podsiadlowski2002}.
Both scenarios could be valid for 4U\,1812--12. However, the lack of hydrogen features in its optical spectrum is difficult to reconcile with these binary solutions. Nevertheless, we note that some transient LMXBs with hydrogen-rich companions did not show H$\alpha$\ emission during some phases of the outburst  (see \citealt{Jimenez-Ibarra2019, Stoop2021} and references therein).

A very appealing alternative for the nature of the system is that 4U\,1812--12\ is a progenitor of an ultra-compact X-ray binary. In this case, the donor star would be an evolved main-sequence star that started mass transfer near or just after the point of central hydrogen exhaustion, and is progressively getting closer to the compact object on its path to the ultra-compact period regime (i.e., the system is in the so-called evolved main-sequence star channel; see \citealt{Nelemans2010b}). In this scenario, traces of hydrogen can still be present in the donor star photosphere ($X_{\rm s}\sim 0.1$), and therefore in the accretion disk \citep{Nelemans2010b, Nelson2003, Podsiadlowski2002}. This scenario would reconcile all the observables of 4U\,1812--12.  First, this solution would be consistent with it being a persistent system, since the mass transfer rate is of the same order as the critical mass transfer rate in an accretion disk with mixed composition ($X=0.1$ and $Y=0.9$, \citealt{Lasota2008}, see Fig.~\ref{fig:DIM}). Second, such a low fraction of hydrogen would not affect the duration of thermonuclear bursts \citep{Cumming2003}, which is in agreement with the numerous short thermonuclear events displayed by the source \citep[][]{Cocchi2000, Galloway2020}. Finally, this low fraction of hydrogen would not be detectable in the optical spectrum \citep{Werner2006}, in agreement with the observations of \citet{ArmasPadilla2020}.

\subsection{An ultra-compact X-ray binary with a 114-min orbital period?}\label{subsec:ucxb}
According to evolutionary models, ultra-compact X-ray binaries can evolve towards orbital periods of $100-110$~min or longer if the donor is heated and inflated, or if the donor’s mass loss via winds is taken into account \citep{VanHaaften2012a, VanHaaften2012b}. However, the predicted mass transfer rate at these longer periods is below $\sim 10^{-12}~\mathrm{M}_{\odot}~\mathrm{yr}^{-1}$, which is two orders of magnitude lower than that of 4U\,1812--12. We note that these computed tracks are for a helium or carbon-oxygen white dwarf companion that fully fills its Roche lobe. The evolutionary tracks for He-star donors presented in \citet{Heinke2013} provide higher mass transfer rates, which may explain the group of persistent ultra-compact systems with orbital periods longer than $\simeq 40$~min and high mass accretion rates. Still, these tracks were calculated only for orbital periods up to 60~min. Thus, the evolution beyond this value is unclear, as is the stage where the He-star core stops expanding.

Setting aside the uncertainties that surround the above He-star evolutionary paths, this tantalizing scenario (i.e., an evolved 114-min ultra-compact system) might be still plausible from a pure stability point of view. Although the mass transfer rate of 4U\,1812--12\ sits below the disk instability lines for both pure He and non-irradiated C/O disks, an irradiated C/O disk would maintain stability if the critical mass transfer rate drops by a similar amount than He and Solar-abundance disks when irradiation is taken into account (i.e., by a factor of $6-10$, see Fig.~\ref{fig:DIM}, \citealt{Menou2002,Lasota2008}). As a matter of fact, evolutionary calculations show that He-star donors can be C/O rich stars with some traces of helium left \citep{Nelemans2010b}. Furthermore, we note that our calculated mass transfer rate is an upper limit, since it does not account for possible mass loss via outflows (see e.g., \citealt{Fender2016, Santisteban2019, Marino2019b}), and therefore the actual mass-transfer rate might still sit above the instability thresholds.

\section{Conclusions}\label{sec:concl}

We have presented time-resolved HiPERCAM/GTC photometry  of the ultra-compact X-ray binary candidate 4U\,1812--12. The $r_\mathrm{s}$$i_\mathrm{s}$$z_\mathrm{s}$ light curves show a clear sawtooth-like periodic modulation that resembles a superhump. An  ANOVA periodogram of the $z_\mathrm{s}$-band light curve favors a period of $\simeq 114$~min. We tentatively propose this as the orbital period of 4U\,1812--12, challenging the previously proposed ultra-compact nature of the binary.

We discuss possible scenarios for the nature of the system.  Based on its properties, namely its persistently dim luminosity, optical spectrum and short thermonuclear type I bursts, we suggest that 4U\,1812--12\ is an ultra-compact X-ray binary progenitor whose orbit is shrinking towards the ultra-compact regime and has an evolved main-sequence star with a low fraction of photospheric hydrogen as donor star.

Additional high-quality observations of 4U\,1812--12\ are desirable in order to verify the persistence of the periodic modulation and confirm the orbital period of the system.

\begin{acknowledgments}
This work is supported by the Spanish Ministry of Science under grants PID2020--120323GB--I00 and EUR2021--122010. We acknowledge support from the Consejería de Economía, Conocimiento y Empleo del Gobierno de Canarias and the European Regional Development Fund (ERDF) under grant with reference ProID2021010132 and ProID2020010104. The design and construction of HiPERCAM was funded by the European Research Council under the European Union’s Seventh Framework Programme (FP/2007--2013) under ERC--2013--ADG Grant Agreement no. 340040 (HiPERCAM). HiPERCAM operations and VSD are supported by STFC grant ST/V000853/1. Based on observations made with the Gran Telescopio Canarias (GTC), installed at the Spanish Observatorio del Roque de los Muchachos of the Instituto de Astrofísica de Canarias, on the island of La Palma. This research made use of Peranso (\url{www.peranso.com}), a light curve and period analysis software.

\end{acknowledgments}

\vspace{5mm}
\facilities{GTC(HiPERCAM)}

\software{ANOVA \citep{schwarzenberg-czerny96-1},
          HiPERCAM pipeline (\url{https://github.com/HiPERCAM/}),
          Peranso (\url{www.peranso.com})
          }

\subsubsection*{#1}}

\pagestyle{headings}
\markright{Reference sheet: \texttt{natbib}}

\usepackage{shortvrb}
\MakeShortVerb{\|}

\begin{document}
\thispagestyle{plain}

\newcommand{\textsc{Bib}\TeX}{\textsc{Bib}\TeX}
\newcommand{\texttt{#1}\def\filedate{#2}\def\fileversion{#3}}}{\texttt{#1}\def\filedate{#2}\def\fileversion{#3}}}
\begin{center}{\bfseries\Large
    Reference sheet for \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}\ usage}\\
    \large(Describing version \fileversion\ from \filedate)
\end{center}

\begin{quote}\slshape
For a more detailed description of the \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}\ package, \LaTeX\ the
source file \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}\texttt{.dtx}.
\end{quote}
\head{Overview}
The \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}\ package is a reimplementation of the \LaTeX\ |\cite| command,
to work with both author--year and numerical citations. It is compatible with
the standard bibliographic style files, such as \texttt{plain.bst}, as well as
with those for \texttt{harvard}, \texttt{apalike}, \texttt{chicago},
\texttt{astron}, \texttt{authordate}, and of course \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}.

\head{Loading}
Load with |\usepackage[|\emph{options}|]{|\texttt{#1}\def\filedate{#2}\def\fileversion{#3}}|}|. See list of
\emph{options} at the end.

\head{Replacement bibliography styles}
I provide three new \texttt{.bst} files to replace the standard \LaTeX\
numerical ones:
\begin{quote}\ttfamily
 plainnat.bst \qquad abbrvnat.bst \qquad unsrtnat.bst
\end{quote}
\head{Basic commands}
The \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}\ package has two basic citation commands, |\citet| and
|\citep| for \emph{textual} and \emph{parenthetical} citations, respectively.
There also exist the starred versions |\citet*| and |\citep*| that print
the full author list, and not just the abbreviated one.
All of these may take one or two optional arguments to add some text before
and after the citation.
\begin{quote}
\begin{tabular}{l@{\quad$\Rightarrow$\quad}l}
  |\citet{jon90}| & Jones et al. (1990)\\
  |\citet[chap.~2]{jon90}| & Jones et al. (1990, chap.~2)\\[0.5ex]
  |\citep{jon90}| & (Jones et al., 1990)\\
  |\citep[chap.~2]{jon90}| & (Jones et al., 1990, chap.~2)\\
  |\citep[see][]{jon90}| & (see Jones et al., 1990)\\
  |\citep[see][chap.~2]{jon90}| & (see Jones et al., 1990, chap.~2)\\[0.5ex]
  |\citet*{jon90}| & Jones, Baker, and Williams (1990)\\
  |\citep*{jon90}| & (Jones, Baker, and Williams, 1990)
\end{tabular}
\end{quote}
\head{Multiple citations}
Multiple citations may be made by including more than one
citation key in the |\cite| command argument.
\begin{quote}
\begin{tabular}{l@{\quad$\Rightarrow$\quad}l}
  |\citet{jon90,jam91}| & Jones et al. (1990); James et al. (1991)\\
  |\citep{jon90,jam91}| & (Jones et al., 1990; James et al. 1991)\\
  |\citep{jon90,jon91}| & (Jones et al., 1990, 1991)\\
  |\citep{jon90a,jon90b}| & (Jones et al., 1990a,b)
\end{tabular}
\end{quote}

\head{Numerical mode}
These examples are for author--year citation mode. In numerical mode, the
results are different.
\begin{quote}
\begin{tabular}{l@{\quad$\Rightarrow$\quad}l}
  |\citet{jon90}| & Jones et al. [21]\\
  |\citet[chap.~2]{jon90}| & Jones et al. [21, chap.~2]\\[0.5ex]
  |\citep{jon90}| & [21]\\
  |\citep[chap.~2]{jon90}| & [21, chap.~2]\\
  |\citep[see][]{jon90}| & [see 21]\\
  |\citep[see][chap.~2]{jon90}| & [see 21, chap.~2]\\[0.5ex]
  |\citep{jon90a,jon90b}| & [21, 32]
\end{tabular}
\end{quote}
\head{Suppressed parentheses}
As an alternative form of citation, |\citealt| is the same as |\citet| but
\emph{without parentheses}. Similarly, |\citealp| is |\citep| without
parentheses. Multiple references, notes, and the starred variants
also exist.
\begin{quote}
\begin{tabular}{l@{\quad$\Rightarrow$\quad}l}
  |\citealt{jon90}| & Jones et al.\ 1990\\
  |\citealt*{jon90}| & Jones, Baker, and Williams 1990\\
  |\citealp{jon90}| & Jones et al., 1990\\
  |\citealp*{jon90}| & Jones, Baker, and Williams, 1990\\
  |\citealp{jon90,jam91}| & Jones et al., 1990; James et al., 1991\\
  |\citealp[pg.~32]{jon90}| & Jones et al., 1990, pg.~32\\
  |\citetext{priv.\ comm.}| & (priv.\ comm.)
\end{tabular}
\end{quote}
The |\citetext| command
allows arbitrary text to be placed in the current citation parentheses.
This may be used in combination with |\citealp|.
\head{Partial citations}
In author--year schemes, it is sometimes desirable to be able to refer to
the authors without the year, or vice versa. This is provided with the
extra commands
\begin{quote}
\begin{tabular}{l@{\quad$\Rightarrow$\quad}l}
  |\citeauthor{jon90}| & Jones et al.\\
  |\citeauthor*{jon90}| & Jones, Baker, and Williams\\
  |\citeyear{jon90}|   & 1990\\
  |\citeyearpar{jon90}| & (1990)
\end{tabular}
\end{quote}
\head{Forcing upper cased names}
If the first author's name contains a \textsl{von} part, such as ``della
Robbia'', then |\citet{dRob98}| produces ``della Robbia (1998)'', even at the
beginning of a sentence. One can force the first letter to be in upper case
with the command |\Citet| instead. Other upper case commands also exist.
\begin{quote}
\begin{tabular}{rl@{\quad$\Rightarrow$\quad}l}
  when & |\citet{dRob98}| & della Robbia (1998) \\
  then & |\Citet{dRob98}| & Della Robbia (1998) \\
     &   |\Citep{dRob98}| & (Della Robbia, 1998) \\
     &   |\Citealt{dRob98}| & Della Robbia 1998 \\
     &   |\Citealp{dRob98}| & Della Robbia, 1998 \\
     &   |\Citeauthor{dRob98}| & Della Robbia
\end{tabular}
\end{quote}
These commands also exist in starred versions for full author names.

\head{Citation aliasing}
Sometimes one wants to refer to a reference with a special designation,
rather than by the authors, i.e. as Paper~I, Paper~II. Such aliases can be
defined and used, textual and/or parenthetical with:
\begin{quote}
\begin{tabular}{lcl}
  |\defcitealias{jon90}{Paper~I}|\\
  |\citetalias{jon90}| & $\Rightarrow$ & Paper~I\\
  |\citepalias{jon90}| & $\Rightarrow$ & (Paper~I)
\end{tabular}
\end{quote}
These citation commands function much like |\citet| and |\citep|: they may
take multiple keys in the argument, may contain notes, and are marked as
hyperlinks.
\head{Selecting citation style and punctuation}
Use the command |\bibpunct| with one optional and 6 mandatory arguments:
\begin{enumerate}
\item the opening bracket symbol, default = (
\item the closing bracket symbol, default = )
\item the punctuation between multiple citations, default = ;
\item the letter `n' for numerical style, or `s' for numerical superscript
      style, any other letter for
      author--year, default = author--year;
\item the punctuation that comes between the author names and the year
\item the punctuation that comes between years or numbers when common author
      lists are suppressed (default = ,);
\end{enumerate}

The optional argument is the character preceding a post-note, default is a
comma plus space. In redefining this character, one must include a space if
one is wanted.

Example~1, |\bibpunct{[}{]}{,}{a}{}{;}| changes the output of
\begin{quote}
  |\citep{jon90,jon91,jam92}|
\end{quote}
into [Jones et al. 1990; 1991, James et al. 1992].

Example~2, |\bibpunct[; ]{(}{)}{,}{a}{}{;}| changes the output of
\begin{quote}
  |\citep[and references therein]{jon90}|
\end{quote}
into (Jones et al. 1990; and references therein).

\head{Other formatting options}
Redefine |\bibsection| to the desired sectioning command for introducing
the list of references. This is normally |\section*| or |\chapter*|.

Define |\bibpreamble| to be any text that is to be printed after the heading but
before the actual list of references.

Define |\bibfont| to be a font declaration, e.g.\ |\small| to apply to
the list of references.

Define |\citenumfont| to be a font declaration or command like |\itshape|
or |\textit|.

Redefine |\bibnumfmt| as a command with an argument to format the numbers in
the list of references. The default definition is |[#1]|.

The indentation after the first line of each reference is given by
|\bibhang|; change this with the |\setlength| command.

The vertical spacing between references is set by |\bibsep|; change this with
the |\setlength| command.

\head{Automatic indexing of citations}
If one wishes to have the citations entered in the \texttt{.idx} indexing
file, it is only necessary to issue |\citeindextrue| at any point in the
document. All following |\cite| commands, of all variations, then insert
the corresponding entry to that file. With |\citeindexfalse|, these
entries will no longer be made.

\head{Use with \texttt{chapterbib} package}

The \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}\ package is compatible with the \texttt{chapterbib} package
which makes it possible to have several bibliographies in one document.

The package makes use of the |\include| command, and each |\include|d file
has its own bibliography.

The order in which the \texttt{chapterbib} and \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}\ packages are loaded
is unimportant.

The \texttt{chapterbib} package provides an option \texttt{sectionbib}
that puts the bibliography in a |\section*| instead of |\chapter*|,
something that makes sense if there is a bibliography in each chapter.
This option will not work when \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}\ is also loaded; instead, add
the option to \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}.

Every |\include|d file must contain its own
|\bibliography| command where the bibliography is to appear. The database
files listed as arguments to this command can be different in each file,
of course. However, what is not so obvious, is that each file must also
contain a |\bibliographystyle| command, \emph{preferably with the same
style  argument}.
\head{Sorting and compressing citations}
Do not use the \texttt{cite} package with \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}; rather use one of the
options \texttt{sort} or \texttt{sort\&compress}.

These also work with author--year citations, making multiple citations appear
in their order in the reference list.

\head{Long author list on first citation}
Use option \texttt{longnamesfirst} to have first citation automatically give
the full list of authors.

Suppress this for certain citations with |\shortcites{|\emph{key-list}|}|,
given before the first citation.

\head{Local configuration}
Any local recoding or definitions can be put in \texttt{#1}\def\filedate{#2}\def\fileversion{#3}}\texttt{.cfg} which
is read in after the main package file.

\head{Options that can be added to \texttt{\char`\\ usepackage}}
\begin{description}
\item[\ttfamily round] (default) for round parentheses;
\item[\ttfamily square] for square brackets;
\item[\ttfamily curly] for curly braces;
\item[\ttfamily angle] for angle brackets;
\item[\ttfamily colon] (default) to separate multiple citations with
     colons;
\item[\ttfamily comma] to use commas as separaters;
\item[\ttfamily authoryear] (default) for author--year citations;
\item[\ttfamily numbers] for numerical citations;
\item[\ttfamily super] for superscripted numerical citations, as in
     \textsl{Nature};
\item[\ttfamily sort] orders multiple citations into the sequence in
     which they appear in the list of references;
\item[\ttfamily sort\&compress] as \texttt{sort} but in addition multiple
     numerical citations are compressed if possible (as 3--6, 15);
\item[\ttfamily longnamesfirst] makes the first citation of any reference
     the equivalent of the starred variant (full author list) and subsequent
     citations normal (abbreviated list);
\item[\ttfamily sectionbib] redefines |\thebibliography| to issue
     |\section*| instead of |\chapter*|; valid only for classes with a
     |\chapter| command; to be used with the \texttt{chapterbib} package;
\item[\ttfamily nonamebreak] keeps all the authors' names in a citation on
     one line; causes overfull hboxes but helps with some \texttt{hyperref}
     problems.
\end{description}
\end{document}