text:
\section{Introduction}\label{introduction}}

The ICH E9(R1) addendum on estimands stresses the importance of a precise description of the targeted treatment
effect reflecting the clinical question posed by the trial objectives \citep{iche9r1}. One important attribute of an estimand is a list of possible intercurrent events (ICEs), i.e.~events occurring after treatment initiation that affect either the interpretation or the existence of the measurements associated
with the clinical question of interest, and the definition of appropriate strategies to deal with ICEs. The two most relevant strategies for the purpose of this article are the hypothetical strategy and the treatment policy strategy. For the hypothetical strategy, a scenario is envisaged in which the ICE would not occur. For the treatment policy strategy, the occurrence of the ICE is considered irrelevant in defining the treatment effect of interest and the value for the outcome of interest is used regardless of whether or not the ICE occurred.

The ICH E9(R1) addendum distinguishes between ICEs and missing data. However, there are many connections between them. Under the hypothetical scenario, outcome values after the ICE are not directly observable and treated using models for missing data. Under a treatment policy scenario, the outcome assessments after the ICE are observable but may be incomplete because study drop out cannot be prevented. In addition, intermittently missing outcome assessments may also occur due to missed visits.

For more than a decade, a likelihood-based mixed effects model repeated measures (MMRM) approach has been the de facto standard for the primary analysis of continuous longitudinal endpoints in pharmaceutical clinical trials \citep{Mallinckrodt2008}. MMRM typically models the longitudinal outcomes at scheduled visits using a multivariate normal model with the visit time treated as a categorical variable in each treatment group, adjustment for baseline covariates, and an unstructured covariance matrix. MMRM models provide valid treatment effect estimates if the missing data are missing at random (MAR) and the complete data model is correctly specified. Informally, this means that missing outcome values can be accounted for by baseline covariates, the assigned treatment group, and observed outcomes. The MAR assumption is often a good starting point for implementing a hypothetical strategy \citep{Mallinckrodt2020}. However, it is typically less plausible when implementing a treatment policy strategy for an ICE such as study treatment discontinuation. For this latter setting as well as for settings which include multiple ICEs handled with different strategies, methods based on imputation of missing data may be more suitable than MMRM \citep{CroEtAlTutorial2020}. For example, it may be feasible to impute missing data after study treatment discontinuation based on subjects from the same treatment group who also discontinued treatment but continued to be followed up \citep[\citet{PolverejanDragalin2020}]{Guizzaro2021}. An alternative which is especially attractive if post-discontinuation data are sparse is to use reference-based imputation methods \citep[\citet{CroEtAlTutorial2020}, \citet{Mallinckrodt2020}]{CarpenterEtAl2013}. These methods formalize the idea to impute missing data in the intervention group based on data from a control or reference group. A typical approach to implement these imputation methods is to use multiple imputation based on Bayesian posterior draws of model parameters combined with Rubin's rules to make inferences.

In this article, conditional mean imputation for MAR and reference-based imputation of missing data is introduced, justified, and explored in a clinial trial application and a simulation study. The proposed approach differs from the conventional multiple imputation methods in several ways. First, it estimates the parameters
of the imputation model using maximum likelihood rather than via Bayesian posterior draws \citep{vanHippelBartlett2021}. Second, it uses a single conditional mean imputation rather than random imputations. Third, inference is based on resampling methods, i.e.~the jackknife or the bootstrap, and targets the repeated sampling variance rather than the variance estimated by Rubin's formula which over-estimates the frequentist variance for reference-based imputation methods \citep[\citet{LiuPang2016}, \citet{Tang2017}, \citet{Bartlett2021}]{Seaman2014}.
Unlike the conventional approach, it avoids the specification of prior distributions and the complexity of Markov chain Monte Carlo (MCMC) sampling and leads to deterministic treatment effect estimates (i.e.~they are free from Monte Carlo sampling error) and, in the case of the jackknife, also to deterministic confidence intervals and $p$-values.

\hypertarget{conditional-mean-imputation-method}{%
\section{Conditional mean imputation method}\label{conditional-mean-imputation-method}}

In brief, the procedure is implemented in four steps. First, a MMRM imputation model is fitted to the observed outcome data. Second, conditional mean imputation of missing data is performed based on the parameter estimates from the MMRM imputation model and the chosen imputation method, e.g.~reference-based imputation. Third, the completed data is analysed using a simple ANCOVA model. Finally, inference is performed based on resampling techniques. All four steps are described and justified below.

\hypertarget{alignment-between-the-estimand-and-the-missing-data-strategy}{%
\subsection{Alignment between the estimand and the missing data strategy}\label{alignment-between-the-estimand-and-the-missing-data-strategy}}

Assume that the data are from a randomized two-group trial of an intervention versus control with $n$ subjects in total and that each subject $i$ ($i=1,\ldots,n$) has $J$ scheduled follow-up visits at which the outcome of interest is assessed.
Denote the observed outcome vector of length $J$ for subject $i$ by $Y_i$ (with missing assessments coded as NA (not available)) and its non-missing and missing components by $Y_{i!}$ and $Y_{i?}$, respectively.

Missing data in $Y_i$ may occur due to missed outcome assessment or may be caused by unobserved counterfactual outcomes relevant to the chosen estimand. Observed outcome data after an ICE handled using a hypothetical strategy is not compatible with this strategy.
Therefore, we assume that all post ICE data after ICEs handled using a hypothetical strategy are already set to NA in $Y_i$. Intermittent missing data and data after such ICEs will be assumed to be MAR (conditional on baseline covariates and observed outcomes). Observed outcome data after an ICE handled using a treatment policy strategy is compatible with the chosen strategy and therefore relevant for the analysis. Such post-ICE data are increasingly systematically collected in RCTs. However, depending on the study context, it may be challenging to retain subjects in the trial after ICEs such as study treatment discontinuation and missing post-ICE data may be common. Two options may be applied for imputation of missing post-ICE data in this setting: MAR imputation after inclusion of time-varying covariates and reference-based imputation methods.

Imputation under a MAR assumption after inclusion of appropriate internal time-varying covariates associated with the ICE (e.g.~time-varying indicators of treatment compliance, discontinuation or initiation of rescue treatment) have been proposed by \citet{Guizzaro2021}. In this approach, post-ICE outcomes are included in every step of the analysis, including in the fitting of the imputation model. The approach allows that ICEs may impact post-ICE outcomes but that otherwise missingness is non-informative. The approach also assumes that deviations in outcomes after the ICE are correctly modeled by these time varying covariates and that sufficient post-ICE data are available to reliably estimate the effect of ICEs on outcome data.

Depending on the disease area and the anticipated mechanism of action of the intervention, it may be plausible to assume that subjects in the intervention group behave similarly to subjects in the control group after the ICE treatment discontinuation. In such settings, reference-based imputation methods are valuable. They are implemented by combining estimated mean trajectories (and covariance matrices) from both treatment groups when determining the imputation distribution for a subject as described in Section \ref{sec:impstep}.

\hypertarget{imputation-model}{%
\subsection{Imputation model}\label{imputation-model}}

The purpose of the imputation model is to estimate mean trajectories and covariance matrices for each treatment group in the absence of ICEs handled using reference-based imputation methods.
Conventionally, publications on reference-based imputation methods have implicitly assumed that the corresponding post-ICE data is missing for all subjects \citep{CarpenterEtAl2013}. We will also allow the situation where post-ICE data is available for some subjects but needs to be imputed using reference-based methods for others. However, any observed data after ICEs for which reference-based imputation methods are specified are not compatible with the imputation model described below and they will therefore be removed and considered as missing for the purpose of estimating the imputation model, and for this purpose only.
If we did not exclude these data, then the imputation model would mistakenly estimate mean trajectories based on a mixture of observed pre- and post-ICE data which are not relevant for reference-based imputations. However, any observed post-ICE data will be included as is in the subsequent imputation and analysis steps. For subjects who experience an ICE that is to be handled by reference-based imputation of missing data, denote the visit after which the ICE occurs by $\tilde{t}_i<J$. For all other subjects, set $\tilde{t}_i=\infty$. A subject's outcome vector after setting observed outcomes after time $\tilde{t}_i$ to \texttt{NA} is denoted as $Y'_i$ and the corresponding data vector after removal of \texttt{NA} elements as $Y'_{i!}$.

The imputation model of the longitudinal outcomes $Y'_i$ assumes that the mean structure is a linear function of covariates. At a minimum, the covariates will include the treatment group, the (categorical) visit, and treatment-by-visit interactions. Typically, other covariates including the baseline outcome will also be added. Interaction terms between the treatment group and baseline covariates as well as three-way interactions between treatment group, baseline covariates, and visit can also be included. Indeed, such interactions are recommended if they are suspected to be present based on clinical considerations \citep{Sullivan2018}.
External time-varying covariates (e.g.~calendar time of the visit) as well as internal time-varying covariates (e.g.~time-varying indicators of treatment discontinuation or initiation of rescue treatment) may in principle also be included if indicated \citep{Guizzaro2021}. Missing covariate values are not allowed. This means that the values of time-varying covariates must be non-missing at every visit regardless of whether the outcome is measured or missing.

Denote the $J\times p$ design matrix for subject $i$ by $X_i$ and the same matrix after removal of rows corresponding to missing outcomes in $Y'_{i!}$ by $X'_{i!}$.
Here $p$ is the number of parameters in the mean structure of the model for the elements of $Y'_{i!}$.
The MMRM imputation model for the observed outcomes is defined as:
\[ Y'_{i!} = X'_{i!}\beta + \epsilon_{i!} \mbox{ with } \epsilon_{i!}\sim N(0,\Sigma_{i!!})\]
where $\beta$ is the vector of regression coefficients and $\Sigma_{i!!}$ is a covariance matrix which is obtained from the complete-data $J\times J$-covariance matrix $\Sigma$ by omitting rows and columns corresponding to missing outcome assessments for subject $i$. Typically, a common unstructured covariance matrix across treatment groups is assumed for $\Sigma$. However, separate covariate structures per treatment group are also possible and recommended in case heteroscedasticity between treatment groups is suspected.

\hypertarget{sec:impstep}{%
\subsection{Conditional mean imputation step}\label{sec:impstep}}

Conditional mean imputation is based on either maximum likelihood or restricted maximum likelihood (REML) estimates of the regression coefficients and covariance matrices from the imputation model which are denoted by $\hat{\beta}$ and $\hat{\Sigma}$. As demonstrated by others, ML-based imputation is often more efficient and avoids the specification of prior distributions and the usage of Markov chain Monte Carlo (MCMC) methods which are required for imputation methods based on Bayesian posterior draws of imputation parameters \citep[\citet{vanHippelBartlett2021}]{Wang1998}.

In order to impute missing data for a subject, the marginal imputation distribution is first defined. For MAR imputation, the marginal imputation distribution for subject $i$ is a multivariate normal distribution with mean $\tilde{\mu}_i=X_i\hat{\beta}$ and covariance $\tilde{\Sigma}_i=\hat{\Sigma}$. For subjects randomized to the control group, the same marginal imputation distribution as for MAR imputation is also used for reference-based imputation methods. For a subject $i$ randomized to the intervention group for whom a reference-based imputation method is used after visit $\tilde{t}_i<J$, denote the predicted mean based on the assigned treatment as $\mu_i=X_i\hat{\beta}$. Denote the corresponding design matrix assuming that (counter to fact) the subject had been randomized to the control group (but using the subject's observed baseline characteristics and time-varying covariates) by $X_{i,ref}$ and denote the corresponding predicted mean from the imputation model by $\mu_{ref,i}=X_{i,ref}\hat{\beta}$. Then, the mean of the marginal imputation distribution depends on the chosen reference-based imputation method \citep[Section 4.3]{CarpenterEtAl2013}:

\begin{enumerate}
\def\arabic{enumi}.{\arabic{enumi}.}
\tightlist
\item
  Copy reference (CR): $\tilde{\mu}_i = \mu_{ref,i}$.
\item
  Jump to reference (J2R): $\tilde{\mu}_i = (\mu_i[1], \dots, \mu_i[\tilde{t}_i], \mu_{ref,i}[\tilde{t}_i+1], \dots, \mu_{ref,i}[J] )^T$.
\item
  Copy increments in reference (CIR): $\tilde{\mu}_i = (\mu_i[1], \dots, \mu_i[\tilde{t}_i], \mu_i[\tilde{t}_i] + (\mu_{ref,i}[\tilde{t}_i+1] - \mu_{ref,i}[\tilde{t}_i)], \dots,\mu_i[\tilde{t}_i]+(\mu_{ref,i}[J] - \mu_{ref,i}[\tilde{t}_i]))^T$.
\end{enumerate}

If the same covariance matrix is assumed for both treatment groups, then the covariance matrix of the marginal imputation distribution for reference-based imputation methods is also equal to $\tilde{\Sigma}_i=\hat{\Sigma}$. Otherwise, the relevant covariance matrix is derived based on the estimated covariance matrices from the intervention and control groups as described in \citet[Section 4.3]{CarpenterEtAl2013}.

Intuitively, the marginal imputation distribution under CR assumes that subjects who discontinue the intervention did not get any benefit from the intervention, J2R assumes that they did get benefit but that the benefit is lost immediately after discontinuation, whereas CIR assumes that the benefit accrued up to discontinuation is retained but that there is no additional residual benefit after the ICE.
In practice, the most conservative imputation method is J2R while CIR is the least conservative.
J2R and CIR may be seen as relatively conservative imputation strategies for a symptomatic or a disease-modifying intervention, respectively. A more general family of reference-based imputation methods has been presented by \citet{White2020causal}.

Imputation of $Y_i$ is then according to the distribution of $Y_{i?}$ conditional on $Y_{i!}$ which is a multivariate normal distribution with mean $E(Y_{i?}\vert Y_{i!})=\tilde{\mu}_{i?} + \tilde{\Sigma}_{i?!} \tilde{\Sigma}_{i!!}^{-1} (Y_{i!} - \tilde{\mu}_{i!})$ where the subscripts ``?'' and ``!'' denote the selection of elements of $\tilde{\mu}_{i}$ and rows and columns of $\tilde{\Sigma}_{i}$ corresponding to missing and observed outcomes of $Y_i$, respectively.
Note that $Y_i$ includes observed post-ICE data, hence they are being conditioned on in the construction of the imputed data. Rather than performing multiple random imputations of the missing data from the conditional distribution of $Y_{i?}\vert Y_{i!}$, we perform a single deterministic conditional mean imputation, i.e.~we impute $Y_{i?}$ by $E(Y_{i?}\vert Y_{i!})$. As demonstrated in the next section, analyzing the deterministically imputed dataset using conditional mean imputation with an ANCOVA model is equivalent to performing infinitely many random imputations, analyzing each randomly imputed dataset using ANCOVA, and then pooling the resulting treatment effect estimates.

\hypertarget{analysis-step}{%
\subsection{Analysis step}\label{analysis-step}}

The imputed dataset using conditional mean imputation (as described above) can be analyzed using an analysis of covariance (ANCOVA) model with the outcome (or the change in the outcome from baseline) at a specific visit $j$ as the dependent variable, the assigned treatment group as the primary covariate and, typically, adjustment for the same baseline covariates as for the imputation model. The treatment effect estimator based on conditional mean imputation is then the regression coefficient corresponding to the treatment assignment and is denoted by $\hat{\theta}_{CMI}$. In many trials, the outcome at the final visit $J$ will be the primary outcome, but the method can also be applied to outcomes from another visit.

This treatment effect estimator is generated conditional on the REML estimate of the imputation model whereas reference-based imputation methods typically rely on Bayesian posterior draws from the imputation model \citep{CarpenterEtAl2013}. As demonstrated in \citet{vanHippelBartlett2021}, point estimates based on REML estimation yield slightly more efficient point estimates than methods based on Bayesian posterior draws.

\citet{vanHippelBartlett2021} further proposed to perform $M$ random imputations based on the REML estimate, analyze them, and use the average of the treatment effect estimates across random imputations as the overall treatment effect estimator. We denote the treatment effect from a single randomly imputed dataset $m$ as $\hat{\theta}_{SI,m}$ and the overall average treatment effect as $\hat{\theta}_{MI,M}=\frac{1}{M}\sum_{m=1}^M \hat{\theta}_{SI,m}$. In contrast, our estimator $\hat{\theta}_{CMI}$ is based on a single imputed dataset using conditional mean imputation.

However, in our particular setting of a continuous outcome analysed by ANCOVA, it is easy to show that the proposed estimator $\hat{\theta}_{CMI}$ is identical to the pooled treatment effect under multiple random imputation with an infinite number of imputations or, more precisely, that
\begin{equation} \label{eq:MiCmiConnection}
\hat{\theta}_{MI,M} \to \hat{\theta}_{CMI} \mbox{  as  } M\to\infty \mbox{ (a.s.)}
\end{equation}
Thus, in this particular modeling setup, the estimator based on conditional mean imputation corresponds to a computationally efficient implementation of the proposal by \citet{vanHippelBartlett2021}.

To prove this, let $Z_m$ and $Z_{CMI}$ correspond to the imputed outcome vector from the visit relevant to the primary endpoint for all subjects based on the $m^{th}$ randomly imputed dataset and conditional mean imputation, respectively. Then
\[\sum_{m=1}^M \frac{1}{M}Z_m\to Z_{CMI} \mbox{  as  } M\to\infty \mbox{  (a.s.)  }\]
by the strong law of large numbers (and because the randomly imputed outcomes $Z_m$ are realizations of the conditional mean imputation plus mean zero noise).
Let $X$ be the design matrix for the ANCOVA analyis model (which is identical for all multiple imputed datasets).
Denote the vector of estimated regression coefficients from the ANCOVA model by $\hat{\beta}_{SI,m}$ and $\hat{\beta}_{CMI}$ for the $m^{th}$ randomly imputed datasets and the conditional mean imputed dataset, respectively.
Then

\[ \hat{\beta}_{MI,M}=\frac{1}{M}\sum_{m=1}^M \hat{\beta}_{SI,m} = \frac{1}{M} \sum_{m=1}^M (X^t X)^{-1}X^t Z_m = (X^t X)^{-1}X^t \sum_{m=1}^M \frac{1}{M}Z_m \to (X^t X)^{-1}X^t Z_{CMI}= \hat{\beta}_{CMI}\]\\
as $M\to\infty$ (a.s.).
This proves \eqref{eq:MiCmiConnection} because the treatment effect estimators $\hat{\theta}_{MI,M}$ and $\hat{\theta}_{CMI}$ correspond to the same component from the regression coefficient vectors $\hat{\beta}_{SI,M}$ and $\hat{\beta}_{CMI}$, respectively.

The proof relies on the fact that the ANCOVA estimator is a linear function of the outcome vector. For complete data, the ANCOVA estimator leads to identical parameter estimates as an MMRM model of all longitudinal outcomes with an arbitrary common covariance structure across treatment groups if covariate-visit interactions are included in the analysis model for all covariates \citep[p.~197]{Amemiya1985}. Hence, the same proof also applies to such MMRM models. We expect that conditional mean imputation is also valid if a general MMRM model is used for the analysis but more involved argument would be required to formally justify this.

\hypertarget{resampling-based-inference-step}{%
\subsection{Resampling-based inference step}\label{resampling-based-inference-step}}

The most frequently used method for inference with multiple imputed data is Rubin's variance formula. However, Rubin's formula is not applicable to imputations methods generated conditional on the REML estimate \citep{vanHippelBartlett2021}. Moreover, Rubin's formula overestimates the repeated sampling variance for reference-based imputation methods \citep[\citet{LiuPang2016}, \citet{Tang2017}, \citet{Bartlett2021}]{Seaman2014}. Rather than deriving complex analytic formulas for inference in our setting, we propose to use resampling methods such as the jackknife and the bootstrap \citep{EfronTibs1994}.

The jackknife requires $n$ estimates of the treatment effect based on data with one subject removed from the original dataset each
time. Denote the treatment effect estimate based on omission of subject $i$ by $\hat{\theta}_{(-i)}$ ($i=1,\ldots,n$).
Then, the jackknife standard error is defined as
\[\hat{se}_{jack}=\left[\frac{(n-1)}{n}\cdot\sum_{i=1}^{n} (\hat{\theta}_{(-i)}-\bar{\theta}_{(.)})^2\right]^{1/2}\]
where $\bar{\theta}_{(.)}$ denotes the mean of all jackknife estimates. Tests of the null hypothesis $H_0: \theta=\theta_0$ (and corresponding confidence intervals) are then based on the $Z$-score $Z=(\hat{\theta}-\theta_0)/\hat{se}_{jack}$ using a normal approximation.

Consistency of the jackknife method for variance estimation has been demonstrated for REML estimators in linear mixed effects models \citep[Chapter 1.4.4]{Jiang2007} and for M estimators \citep[Example 2.5]{Shao1995jackknife} under suitable regularity conditions.
As described, the conditional mean imputation is a simple transformation of the REML estimator of the imputation model, the observed data, and missing data indicators, and the ANCOVA analysis model is a linear transformation of the imputed outcomes.
Therefore, our procedure can also be represented as an M-estimator, i.e. as a stacked set of estimating equations for estimating the variance components and parameters indexing the mean function of the MMRM model and those for the analysis model.

Bootstrap methods can be used as an alternative to the jackknife and have already been recommended for inference with multiple imputed missing data \citep[\citet{BartlettHughes2020}]{vanHippelBartlett2021}. In most trials, the sample size $n$ should be sufficiently large to ensure that bootstrap standard errors and normal approximations are reliable. An advantage of the bootstrap over the jackknife is that if there are concerns about the adequacy of the normal approximation, then more precise bootstrap confidence intervals (e.g.~percentile or BCa intervals) and tests can be used for inference instead \citep{EfronTibs1994}. The disadvantages of the bootstrap are that the inference is no longer deterministic and that it is more computationally intensive as the required number of bootstrap samples $B$ is typically much larger than the sample size $n$. As a minimum, $B=999$ is recommended \citep[page 202]{DavisonHinkley1997} and, as described in the \textcolor{blue}{Appendix}, substantially higher values may be required to obtain accurate $p$-values which are minimally affected by Monte Carlo error.

\hypertarget{application}{%
\section{Application}\label{application}}

We applied the proposed methods to a publicly available example data set from an antidepressant clinical trial of an active drug versus placebo which was also used in \citet{Mallinckrodt2013} and \citet{Tang2017}. The relevant endpoint is the Hamilton 17-item rating scale for depression (HAMD17) which was assessed at baseline and at weeks 1, 2, 4, and 6. Study drug discontinuation occurred in 24\% (20/84) for the active drug and 26\% (23/88) for placebo. All data after study drug discontinuation were missing and there was a single additional intermittent missing observation. The imputation model had the mean change from baseline in the HAMD17 score as the outcome, included the treatment group, the (categorical) visit, treatment-by-visit interactions, the baseline HAMD17 score, and baseline HAMD17-by-visit interactions as covariates, and assumed a common unstructured covariance matrix in both groups.
The dataset included one intermittent missing value which was imputed under a MAR assumption. For imputing missing data after study drug discontinuation, we explored MAR and reference-based imputation methods. The ANCOVA analysis model also adjusted for the baseline HAMD17 value. The results for conventional Bayesian multiple imputation combined with Rubin's rule (with $M$=1,000 imputations) and conditional mean imputation using jackknife or bootstrap (with $B$=10,000 bootstrap samples) standard errors, respectively, are reported in Table 1.

\begin{table}

\caption{\label{tab:unnamed-chunk-2}Results for the data set from an antidepressant clinical trial.}
\centering
\begin{tabular}[t]{lllllll}
\toprule
\multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{2}{c}{LS mean changes} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} & \multicolumn{1}{c}{} \\
\cmidrule(l{3pt}r{3pt}){3-4}
Imputation & Method & Drug & Placebo & Difference & Standard error & Two-sided $p$-value\\
\midrule
MAR & Bayes + Rubin & -7.64 & -4.84 & 2.80 & 1.11 & 0.013\\
 & Conditional mean &  &  &  &  & \\
 & - Jackknife & -7.64 & -4.83  & 2.80 & 1.11 & 0.011\\
 & - Bootstrap & -7.64 & -4.83  & 2.80 & 1.09 & 0.010 \vspace{.1in} \\

J2R & Bayes + Rubin & -6.96 & -4.84 & 2.12 & 1.12 & 0.060\\
 & Conditional mean &  &  &  &  & \\
 & - Jackknife & -6.96 & -4.84 & 2.13 & 0.86 & 0.013\\
 & - Bootstrap & -6.96 & -4.84 & 2.13 & 0.85 & 0.012 \vspace{.1in} \\

CR & Bayes + Rubin & -7.21 & -4.85 & 2.36 & 1.10 & 0.034\\
 & Conditional mean &  &  &  &  & \\
 & - Jackknife & -7.21 & -4.84 & 2.37 & 0.98 & 0.016\\
 & - Bootstrap & -7.21 & -4.84 & 2.37 & 0.97 & 0.014 \vspace{.1in} \\

CIR & Bayes + Rubin & -7.29 & -4.84 & 2.45 & 1.10 & 0.028\\
 & Conditional mean &  &  &  &  & \\
 & - Jackknife & -7.28 & -4.84 & 2.45  & 1.00 & 0.014\\
 & - Bootstrap & -7.28 & -4.84 & 2.45 & 0.99 & 0.013\\
\bottomrule
\end{tabular}
\end{table}

As shown in Table 1, conventional Bayesian multiple imputation methods and conditional mean imputation result in nearly identical estimates of the treatment effect for all imputation approaches. However, for reference-based imputation methods, the resampling-based standard errors of the treatment effect from conditional mean imputations are considerably smaller than those from Rubin's rule. The reported treatment effects estimates and standard errors are very similar to those obtained in \citet{Tang2017} using a different implementation. The treatment effect estimate and the standard error for conditional mean imputation were smallest for J2R and the bootstrap produced slightly smaller standard error estimates than the jackknife for all methods. The estimated computing time to obtain all four reported treatment effect estimates and corresponding standard errors with our R software implementation were 14.48 minutes for Bayesian multiple imputation ($M=1,000$), 5.16 minutes for conditional mean imputation plus jackknifing, and 312 minutes (5.2 hours) for conditional mean imputation plus bootstrapping ($B=10,000$, i.e. on average 1.87 seconds per bootstrap sample) on a single core of a low-power laptop with an Intel Core i5-8365U processor (1.60 GHz), 16 GB RAM, and Windows 10 Enterprise.

\hypertarget{simulation-study}{%
\section{Simulation study}\label{simulation-study}}

To further explore the statistical properties of the methods, we conducted a simulation study of a trial of an active drug versus placebo with 100 subjects per group and bi-monthly visits from baseline until 12 months. The other simulation parameters were chosen as follows:

\begin{itemize}
\tightlist
\item
  The mean outcome trajectory in the placebo group (and in the active group under the null hypothesis) increased linearly from 50 at baseline to 60 at 12 months.
\item
  The mean outcome trajectory in the active group under the alternative hypothesis was identical to the placebo group up to month 4. From month 4 onward, the slope decreased by 50\% to 5 points/year.
\item
  The covariance structure of the baseline and follow-up values in both groups was implied by a random intercept and slope model with a standard deviation of 5 for both the intercept and the slope, and a correlation of 0.25. In addition, an independent residual error with standard deviation 2.5 was added to each assessment. This implies marginal standard deviations ranging from 5.59 at baseline to 8.29 at the last visit in both groups.
\item
  The probability of study drug discontinuation after each visit was calculated according to a logistic model which also depended on the observed outcome at that visit. Specifically, a visit-wise discontinuation probability of 1.5\% and 2.5\% in the placebo and active group, respectively, was specified if the observed outcome was 50 or lower. For outcomes above 50, the odds of discontinuation further increase by 50\% for each 10 point increase.
\item
  Study drug discontinuation had no effect on the mean trajectory in the placebo group. In the active group, subjects who discontinued followed the slope of the mean trajectory from the placebo group from that time point onward (CIR).
\item
  Study drop-out at the study drug discontinuation visit occurred with a probability of 75\% leading to missing outcome data from that time point onward.
\end{itemize}

Under the alternative, the simulation parameters imply a true treatment effect of -2.59 (7.41 vs 10.00) under a treatment policy strategy which is the target of this simulation study. The treatment discontinuation probabilities were 34\% and 24\% in the active and placebo groups, respectively, under the null hypothesis and 31\% and 24\% under the alternative hypothesis.

Reference-based imputation methods of missing data were compared to MAR imputation. Observed post-discontinuation data was included in the analysis model but excluded from the imputation model for both reference-based and MAR-based imputation methods.
Imputation and analysis models were as described for the antidepressant clinical trial application. Results for conventional Bayesian multiple imputation combined with Rubin's rule were based on $M=100$ random imputation and bootstrap standard errors were calculated for $B$=1,000 bootstrap samples per simulation scenario.

Results of the simulations under the null hypothesis based on 100,000 simulated data sets are presented in Table 2.
This choice of the number of simulations provides a Monte Carlo standard error for type I error estimates of approximately 0.07\%.
As expected, the mean treatment effect estimate was 0 for all imputation methods and scenarios. For Bayesian multiple imputation, the observed standard deviation of treatment effect estimates across simulation runs ($SD$ $\hat{\theta}$) was substantially smaller than the mean treatment effect standard errors from Rubin's rule (mean $\hat{se}_{\hat{\theta}}$) for reference-based imputation methods. Consequently, the simulated type I error ranged from 0.9\%-2.5\% and was substantially below the nominal type I error of 5\%. For conditional mean imputation approaches, there was close agreement between the repeated sampling standard deviation $SD$ $\hat{\theta}$ and the average standard error. The simulated type I error for the jackknife ranged from 4.8\% to 5.0\% which was slightly below but very close to the targeted type I error of 5\%. Inference based on bootstrap standard errors was associated with a small type I error inflation with simulated type I error ranging from 5.1\% to 5.3\%.

It is sometimes reported that MMRM models such as our imputation model using an unstructured covariance matrix result in convergence problems \citep{Mallinckrodt2008}. In this case, a simplified correlation structure, e.g.~a Toeplitz or autoregressive structure, may be applied. In our simulations with a relatively small sample size and a substantial proportion of subjects with treatment discontinuations and missing data, the imputation model failed to converge in only 8/100,000 (0.008\%) simulated datasets and these were excluded from our summaries in Table 1. Jackknifing did not result in any additional convergence problems and convergence problems occurred only for 1/((100,000-8)*1,000) bootstrap samples. In case convergence problems occurred for a bootstrap sample, we replaced it by a new bootstrap sample which seems unproblematic due to the minimal number of observed convergence problems.

\begin{table}

\caption{\label{tab:unnamed-chunk-3}Results from the simulation study under the null hypothesis (for a nominal two-sided significance level of 5\%).}
\centering
\begin{tabular}[t]{llrrr}
\toprule
Imputation & Parameter & Bayes+Rubin & \makecell[r]{Conditional mean\\ (jackknife)} & \makecell[r]{Conditional mean\\ (bootstrap)}\\
\midrule
MAR & Mean $\hat{\theta}$ & 0.00 & 0.00 & 0.00\\
 & SD $\hat{\theta}$ & 0.93 & 0.93 & 0.93\\
 & Mean $\hat{se}_{\hat{\theta}}$ & 0.92 & 0.93 & 0.92\\
 & Type I error & 5.0\% & 4.9\% & 5.3\%\vspace{.1in} \\

J2R & Mean $\hat{\theta}$ & 0.00 & 0.00 & 0.00\\
 & SD $\hat{\theta}$ & 0.69 & 0.69 & 0.69\\
 & Mean $\hat{se}_{\hat{\theta}}$ & 0.93 & 0.69 & 0.69\\
 & Type I error & 0.9\% & 4.8\% & 5.1\%\vspace{.1in} \\

CR & Mean $\hat{\theta}$ & 0.00 & 0.00 & 0.00\\
 & SD $\hat{\theta}$ & 0.80 & 0.80 & 0.80\\
 & Mean $\hat{se}_{\hat{\theta}}$ & 0.91 & 0.81 & 0.80\\
 & Type I error & 2.5\% & 5.0\% & 5.2\%\vspace{.1in} \\

CIR & Mean $\hat{\theta}$ & 0.00 & 0.00 & 0.00\\
 & SD $\hat{\theta}$ & 0.80 & 0.80 & 0.80\\
 & Mean $\hat{se}_{\hat{\theta}}$ & 0.92 & 0.80 & 0.79\\
 & Type I error & 2.4\% & 4.9\% & 5.2\%\\
\bottomrule
\end{tabular}
\end{table}

Results of the simulations under the alternative hypothesis based on 10,000 simulated data sets (providing a Monte Carlo standard error for power estimates of \textless0.5\%) are reported in Table 3. The mean treatment effect estimate under a CIR assumption was identical to the simulation truth of -2.59 for all three methods. MAR imputation gave inflated treatment effect estimates whereas CR and J2R were conservative. Rubin's rule overestimated the frequentist standard error for all reference-based methods leading to a substantial reduction in the estimated power compared to methods based on conditional mean imputation plus re-sampling. The frequentist standard error was lowest for J2R. Surprisingly, this led to a larger simulated power for J2R than for CIR or CR. Inference based on bootstrapping led to small power gains of 0.3-0.4\% across the four scenarios compared to the jackknife.

\begin{table}

\caption{\label{tab:unnamed-chunk-4}Results from the simulation study under the alternative hypothesis (for a nominal two-sided significance level of 5\%).}
\centering
\begin{tabular}[t]{llrrr}
\toprule
Method & Parameter & Bayes+Rubin & \makecell[r]{Conditional mean\\ (jackknife)} & \makecell[r]{Conditional mean\\ (bootstrap)}\\
\midrule
MAR & Mean $\hat{\theta}$ & -3.14 & -3.14 & -3.14\\
 & SD $\hat{\theta}$ & 0.93 & 0.92 & 0.92\\
 & Mean $\hat{se}_{\hat{\theta}}$ & 0.93 & 0.93 & 0.92\\
 & Power & 92.1\% & 92.1\% & 92.4\%\vspace{.1in} \\

J2R & Mean $\hat{\theta}$ & -2.38 & -2.38 & -2.38\\
 & SD $\hat{\theta}$ & 0.74 & 0.74 & 0.74\\
 & Mean $\hat{se}_{\hat{\theta}}$ & 0.95 & 0.75 & 0.74\\
 & Power & 74.9\% & 90.4\% & 90.7\%\vspace{.1in}\\

CR & Mean $\hat{\theta}$ & -2.55 & -2.55 & -2.55\\
 & SD $\hat{\theta}$ & 0.83 & 0.83 & 0.83\\
 & Mean $\hat{se}_{\hat{\theta}}$ & 0.93 & 0.84 & 0.83\\
 & Power & 80.4\% & 86.7\% & 87.1\%\vspace{.1in}\\

CIR & Mean $\hat{\theta}$ & -2.59 & -2.59 & -2.59\\
 & SD $\hat{\theta}$ & 0.82 & 0.82 & 0.82\\
 & Mean $\hat{se}_{\hat{\theta}}$ & 0.93 & 0.83 & 0.82\\
 & Power & 82.0\% & 88.3\% & 88.7\%\\
\bottomrule
\end{tabular}
\end{table}

\hypertarget{discussion}{%
\section{Discussion}\label{discussion}}

We introduced and justified conditional mean imputation based on the REML estimates of the parameters of the imputation model for MAR and reference-based imputation methods with longitudinal multivariate normal outcomes. For inference, we explored the use of the jackknife and the bootstrap.
The data example and the simulations illustrated that this leads to nearly identical treatment effect estimates compared to established methods based on multiple imputation using Bayesian posterior draws form the imputation model parameters and Rubin's rules. The resampling methods produced standard errors which were consistent with the observed repeated sampling variation. The jackknife demonstrated exact protection of the type I error in simulations with a relatively low sample size ($n=100$ per group) and a substantial amount of missing data (\textgreater25\% of subjects with treatment discontinuations) whereas the bootstrap based on a normal approximation showed a slightly increased type I error rate compared to the nominal value (up to 5.3\% at a nominal 5\% significance level). The method based on Bayesian posterior draws and Rubin's rule had inflated standard errors relative to the true repeated sampling variance, very conservative observed type I error rates, and a substantially decreased statistical power for reference-based imputation methods.

It is well known that Rubin's rule does not provide frequentist consistent estimates of the standard error for reference-based imputation methods \citep[\citet{LiuPang2016}, \citet{Tang2017}, \citet{CroEtAl2019}, \citet{Bartlett2021}]{Seaman2014}. Intuitively, this occurs because reference-based imputation methods borrow information from the reference group for imputations in the intervention group. This creates a positive correlation between outcomes in the two treatment groups and leads to a reduction in the frequentist variance of the resulting treatment effect contrast which is not captured by Rubin's variance estimator. Formally, the discrepancy is due to uncongeniality between the imputation and analysis models in reference-based imputation methods \citep[\citet{Bartlett2021}]{Meng1994}.
\citet{CroEtAl2019} argued that Rubin's rule is nevertheless valid for reference-based imputation methods because it is approximately information-anchored, i.e.~standard error estimates from reference-based imputation methods are similar to those observed under MAR imputation. In contrast, several other authors have implicitly or explicitly favored inference that is correct from a frequentist repeated-sampling perspective \citep[\citet{LiuPang2016}, \citet{Tang2017}, \citet{Bartlett2021}]{Seaman2014}.
To us, information anchoring is a sensible concept for sensitivity analyses, whereas for a primary analyses, we feel that that it is more important to adhere to the principles of frequentist inference. Analyses of data with missing observations generally rely on unverifiable missing data assumptions and the assumptions for reference-based imputation methods are relatively strong. Therefore, these assumptions need to be clinically justified as appropriate or at least conservative for the considered disease area and the anticipated mechanism of action of the intervention. Furthermore, appropriate sensitivity analyses should be planned. While we have not explicitly discussed it, $\delta$-based imputation approaches which adjust imputed datasets prior to the analysis for sensitivity analysis purposes could also be incorporated in our framework in a straightforward manner \citep{CroEtAlTutorial2020}.

The proposed approach requires resampling for inference which is generally applicable but also computationally intensive.
However, the time-consuming resampling procedure can easily be parallelized and computing time drastically reduced if computations are distributed over multiple cores.
For special situations, closed-form formulas for standard errors of reference-based imputation methods have been presented by \citet{LiuPang2016} and \citet{Tang2017} but extending these formulas to more general settings (e.g.~allowing for different covariance structures in the two treatment groups or accounting for the inclusion of post-ICE data as proposed here) would be very challenging. Based on our simulations, we recommend the jackknife over the bootstrap for inference because it performed better in our simulation study and is typically much faster to compute than the bootstrap. In both the data example and the simulation study, jackknife standard errors tended to be slightly larger than bootstrap standard errors. This is in line with theoretical investigations which suggest that the jackknife variance tends to be biased upwards for finite sample sizes \citep{Efron1981} whereas bootstrap variances are more susceptible to downwards bias \citep[Section 3.4.3]{Shao1995jackknife}. As a consequence, the jackknife demonstrated exact type I error protection whereas the bootstrap led to slight type I error inflation.
If the bootstrap is applied, we recommend a large number of bootstrap samples (at least $B=10,000$) and, especially for smaller sample sizes, the usage of more accurate methods for confidence intervals and testing than the normal approximation based on the bootstrap standard error.
A further advantage of the jackknife is that the proposed conditional mean imputation approach leads to deterministic treatment effect estimates, standard errors, and inference. This is particularly important in a regulatory setting where it is important to ascertain whether a calculated $p$-value which is close to the critical boundary of 5\% is truly below or above that threshold rather than being uncertain about this because of Monte Carlo error.

In summary, we believe that conditional mean imputation for MAR and reference-based imputation of missing data combined with the jackknife is a useful complementary approach to Bayesian multiple imputation for longitudinal multivariate normal outcomes. It provides replicable parameter estimates and standard errors and correct frequentist inference.

\hypertarget{software-implementation}{%
\section*{Software implementation}\label{software-implementation}}

All presented methods have been implemented in an R package `rbmi` which will be uploaded to CRAN in early 2022.
The data analysis of the antidepressant clinical trial is described in the `Quick start` vignette of the package.
All simulation code has been uploaded to github and can be found at \url{https://github.com/nociale/Simulation_condmean_manuscript}.

\hypertarget{acknowledgement}{%
\section*{Acknowledgement}\label{acknowledgement}}

The authors thank Rachid Abbas for helpful remarks on a draft of the manuscript.

\hypertarget{funding-and-disclosure-statement}{%
\section*{Funding and disclosure statement}\label{funding-and-disclosure-statement}}

Bartlett's contribution was supported by the UK Medical Research Council (Grant MR/T023953/1).

Bartlett's institution has received consultancy fees for the author's advice on statistical methodology from AstraZeneca, Bayer, Novartis, and Roche. Bartlett has received consultancy and course fees from Bayer and Roche.

\hypertarget{a-appendix-additional-details-on-bootstrap-methods}{%
\section*{A Appendix: Additional details on bootstrap methods}\label{a-appendix-additional-details-on-bootstrap-methods}}
\addcontentsline{toc}{section}{A Appendix: Additional details on bootstrap methods}

\hypertarget{a.1-bootstrap-confidence-intervals-ci-and-tests-for-the-treatment-effect}{%
\subsection*{A.1 Bootstrap confidence intervals (CI) and tests for the treatment effect}\label{a.1-bootstrap-confidence-intervals-ci-and-tests-for-the-treatment-effect}}
\addcontentsline{toc}{subsection}{A.1 Bootstrap confidence intervals (CI) and tests for the treatment effect}

A large number of bootstrap methods for CI estimation and testing have been proposed in the literature. We first discuss methods based on the bootstrap standard error and the normal approximation and percentile bootstrap methods \citep[\citet{DavisonHinkley1997}]{EfronTibs1994}.

Denote the treatment effect estimates from $B$ bootstrap samples by $\hat{\theta}^*_b$ ($b=1,\ldots,B$). The \emph{bootstrap standard error} $\hat{se}_{boot}$ is defined as the empirical standard deviation of the bootstrapped treatment effect estimates. The corresponding two-sided normal approximation $1-\alpha$ CI is defined as $\hat{\theta}\pm z^{1-\alpha/2}\cdot \hat{se}_{boot}$ where $\hat{\theta}$ is the treatment effect estimate in the original dataset, i.e.~$\hat{\theta}=\hat{\theta}_{CMI}$. Tests of the null hypothesis $H_0: \theta=\theta_0$ are then based on the $Z$-score $Z=(\hat{\theta}-\theta_0)/\hat{se}_{boot}$ using a standard normal approximation.

Alternatively, \emph{percentile bootstrap} methods can be used for inference: For this method, the $1-\alpha$ CI is defined as the interval from the $(B+1)\alpha/2^{th}$ to the $(B+1)(1-\alpha/2)^{th}$ ordered bootstrap treatment effect estimate \citep[page 202f]{DavisonHinkley1997}.
Typically, $B$ is chosen such that $(B+1)\alpha/2$ is an integer value. If this is not the case, the corresponding empirical bootstrap quantiles can be calculated via interpolation. The advantage of the percentile method over the standard normal approximation is that it is also valid if the treatment effect estimator itself is not normally distributed around the population treatment effect but if a monotone transformation exists which normalizes the estimator \citep[page 173f]{EfronTibs1994}. For example, this is the case for $t$-distributed test statistics.

A \emph{one-sided} bootstrap $p$-value for a test of the null hypothesis $H_0: \theta=\theta_0$ versus the alternative $H_1:\theta > \theta_0$ can be defined via inversion of the percentile CI as $\hat{p}= (\#\{\hat{\theta}^*_b < \theta_0\}+1)/(B+1)$ \citetext{\citealp[page 214-215]{EfronTibs1994}; \citealp[adding +1 to comply with the percentile bootstrap definition of][]{DavisonHinkley1997}}. The \emph{one-sided} test in the other direction can be defined in the same way, and the corresponding two-sided $p$-value is defined as 2 times the lower of the two one-sided $p$-values.

More complex bootstrap confidence intervals also exist and the BCa (bias-corrected and accelerated) CI is often recommended \citep{Carpenter2000}. This would be also possible but not trivial in our setting: Typically, $a$ is estimated based on the skewness of the score function \citep[page 205]{DavisonHinkley1997} (which is not available in closed form in our setting) or on a jackknife approximation to it \citep{Carpenter2000} (which is computationally intensive in our setting).

Permutation tests and associated bootstrap tests which draw bootstrap samples under the null hypothesis of equality between the two randomized treatment groups \citetext{\citealp[chapters 15 and 16]{EfronTibs1994}; \citealp[chapter 4]{DavisonHinkley1997}} could also be applied but have not been explored in our setting.

\hypertarget{a.2-how-many-bootstrap-samples}{%
\subsection*{A.2 How many bootstrap samples?}\label{a.2-how-many-bootstrap-samples}}
\addcontentsline{toc}{subsection}{A.2 How many bootstrap samples?}

As a general rule of thumb, \citet{DavisonHinkley1997} (pages 156 and 202) advise to use $B=999$ or more for CI estimation and testing. However, as illustrated below, substantially larger values of $B$ may be required for accurate estimation of $p$-values. Let $\hat{p}_{\infty}$ denote the one-sided $p$-value that would be obtained under the theoretical bootstrap distribution (including all possible bootstrap samples) and $\hat{p}_{B}$ the corresponding $p$-value based on $B$ bootstrap samples.
The approximate distribution of $\hat{p}_{B}$ is:

\begin{itemize}
\tightlist
\item
  $\hat{p}_{B}\sim \Phi(\Phi^{-1}(\hat{p}_{\infty})/\sqrt{Z/(B-1)})$ with $Z\sim\chi^2(df=B-1)$ for the normal approximation
\item
  $\hat{p}_{B}\sim (Z+1)/(B+1)$ with $Z\sim Bin(B,\hat{p}_{\infty})$ for the inverted bootstrap percentile method
\end{itemize}

Based on these approximations, the probabilities below were evaluated:

\begin{tabular}{l|l|l|l}
\hline
\bf{Method} & \bf{B=999} & \bf{B=9,999} & \bf{B=99,999}\\
\hline
\bf{Normal approximation} &  &  & \\
\hline
95\% range for $\hat{p}_B$ if $\hat{p}_{\infty}=2.5\%$ & 2.02\% to 3.02\% & 2.34\% to 2.66\% & 2.45\% to 2.55\%\\
\hline
Probability of $\hat{p}_B\leq 2.5\%$ if $\hat{p}_{\infty}=2\%$ & 98.37\% & 100.00\% & 100.00\%\\
\hline
Probability of $\hat{p}_B > 2.5\%$ if $\hat{p}_{\infty}=3\%$ & 96.42\% & 100.00\% & 100.00\%\\
\hline
\bf{Percentiles method} &  &  & \\
\hline
95\% range for $\hat{p}_B$ if $\hat{p}_{\infty}=2.5\%$ & 1.70\% to 3.60\% &  2.21\% to 2.82\% & 2.40\% to 2.60\%\\
\hline
Probability of $\hat{p}_B\leq 2.5\%$ if $\hat{p}_{\infty}=2\%$ & 84.66\% & 99.97\% & 100.00\%\\
\hline
Probability of $\hat{p}_B > 2.5\%$ if $\hat{p}_{\infty}=3\%$ & 84.54\% & 99.88\% & 100.00\%\\
\hline
\end{tabular}

These tables demonstrate that the approximation of the bootstrap distribution with a finite number of bootstrap samples $B$ requires a larger $B$ for high accuracy of the percentile method (which is valid under weaker assumptions) than for the method based on the bootstrap standard error (which relies on a normal approximation).

redpajama_set_name: RedPajamaArXiv


text:
import { Component } from '@stencil/core';

@Component({
  tag: 'notfound-page',
  styleUrl: 'notfound-page.scss'
})
export class NotFoundPage {

  render() {
    return (

Woops! We can't find the page your looking for.

Head on back to the <stencil-route-link url="/" class="block">Icons page</stencil-route-link>.

    );
  }
}

redpajama_set_name: RedPajamaGithub


text:
Manager, FP&A - Sales
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Have you ever found a new favourite series on Netflix, picked up groceries curbside at Walmart, or paid for something using Square? That's the power of data in motion in action—giving organisations instant access to the massive amounts of data that is constantly flowing throughout their business. At Confluent, we're building the foundational platform for this new paradigm of data infrastructure. Our cloud-native offering is designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organisation. With Confluent, organisations can create a central nervous system to innovate and win in a digital-first world.
We're looking for self-motivated team members who crave a challenge and feel energised to roll up their sleeves and help realise Confluent's enormous potential. Chart your own path and take healthy risks as we solve big problems together. We value having diverse teams and want you to grow as we grow—whether you're just starting out in your career or managing a large team, you'll be amazed at the magnitude of your impact.
We are seeking an experienced and self-motivated finance manager to join our Sales FP&A team leading the APAC region. In this high-profile role you will work with executive business partners to help drive strategic decision-making, provide financial insight, and build operational rigor and efficiency. You will help guide the financial planning, forecasting, and reporting for these teams and provide recommendations to your business partners within these key functional areas. You will also be responsible for key financial deliverables and ad hoc analyses and help to define the processes by which we scale and grow these teams.
This role will report to the international sales Director, FP&A and will be based in London.
Provide thought leadership to Sales to drive top-line and expense strategic initiatives, and help develop both short-term and long-term policies
Partner with Sales leadership to develop a territory planning model and growth plan for our domestic and international markets
Build / Drive sales capacity models to support long range planning & bookings growth
Master key customer metrics, such as LTV, CAC, and Cohort Analysis as well as management reporting packages, such as sales key metrics and KPIs
Strong controls on top to bottom P&L metrics with focus on OPEX and headcount
Identify and lead initiatives to continuously improve, automate, and scale forecasting and reporting tools, processes, and methodologies
Prior Sales FP&A experience at a high growth subscription software company
Ability to influence business partners and provide recommendations in order to drive decision-making and execution
Experience developing and improving processes with desire to inspire change across the org
Advanced Excel and PowerPoint skills; must also be proficient using G-Suite
Strong communication skills (written, verbal & presentation) with the ability to understand and communicate difficult concepts
Excellent financial modelling and analysis skills
Ability to work quickly and manage multiple tasks in a fast-paced, deadline-driven environment while maintaining a high level of precision and attention to details
Demonstrated an extraordinary work ethic
A highly motivated team player and experience with people management
Additional highly desirable experience
Experience with Salesforce, NetSuite, and Anaplan
Experience in subscription/consumption environments
At Confluent, equality is a core tenet of our culture. We are committed to building an inclusive global team that represents a variety of backgrounds, perspectives, beliefs, and experiences. The more diverse we are, the richer our community and the broader our impact.
Click here to review our California Candidate Privacy Notice, which describes how and when Confluent, Inc., and its group companies, collects, uses, and shares certain personal information of California job applicants and prospective employees.

redpajama_set_name: RedPajamaCommonCrawl


The danger of isolating Russia from the West was the theme of Matthew Del Santo's post on the ABC's The Drum site on 4 August 2014, a hundred years after the July Crisis that precipitated the First World War.
"On the threshold of a century that will test the West's 500-year global dominance, the West would be scoring a massive own-goal by pushing Russia and China into an alliance," wrote Del Santo, who is a former officer of our Department of Foreign Affairs and Trade.
"Today (2014) as China's growing power drives America ever deeper into a great-power standoff in East Asia, the West's tone-deaf policies in Western Eurasia risk midwifing a Sino-Russia alliance of the contained and sanctioned – a Central Powers 2.0 on a hemispheric scale ….
A fascinating wartime perspective on Russian relations with Western Europe comes to us from Hungarian intellectual Karl Polanyi in the essay – Why make Russia run amok? – written for the March 1943 edition of Harper's Magazine and reproduced in a new Polanyi selection edited by Michele Cangiani and Claus Thomasberger, published by Polity.
Polanyi harks back to the Four Power Pact of England, France, Italy and Germany first mooted in public by Mussolini on 17 March 1933 and quickly signed off with Mussolini by British Prime Minister Ramsay MacDonald and Foreign Secretary Sir John Simons when they flew to Rome. This was a Concert of Europe designed to replace The League of Nations. "It would not be idealistic but at least it might work." Russia was excluded.
"The only revolution the City of London had ever understood was the French Revolution of 1789. Since the German Revolution of 1933 did not resemble it a bit, the City was reassured that it was not a revolution. On the other hand, the Russian Revolution of 1917 not only resembled the French but was in many details a veritable copy of it. Who but a fool could doubt which of the two was the enemy?
Polanyi traces the ancestry of Munich to the earlier crises of Manchuria and Ethiopia where the Russians could have intervened effectively, if they had been invited, and to the final catastrophe of Spain.
Polanyi could easily lead us to debate about the origins of The Cold War and that would have to involve a trip to Bretton Woods in 1944 where those collegial protagonists John Maynard Keynes and Harry Dexter White planned the post-war world a month after the Normandy landings. White wanted to bring Russia into the club. Four years later White died under a cloud of suspicion that he was a Russian agent. It is interesting and relevant but we had better skip back to the present.
In Warsaw on 24 October – just a couple of weeks ago – Lieutenant General Ben Hodges predicted that America would be at war with China within 15 years. It was not inevitable but war was a very strong likelihood, said the recently retired Commander of the US Army in Europe.
The 15 years are needed in the view of American service chiefs to build up sufficient strength to fight simultaneous wars in Europe and the Pacific as the Americans did with such phenomenal economic power between 1942 and 1945.
There are two deep flaws in the strategy. The Chinese grow stronger every year and 15 years hence will be truly formidable. Secondly, there is no guarantee the next three American administrations will build up defence budgets in the manner to which the Pentagon has become accustomed. If the Anglospheric establishment is determined to knock China down to size a time frame of 15 months makes more sense that 15 years.
Donald Trump won office in 2016 with three excellent policies – to stop baiting the Russian bear, to stop blowing the Middle East to bits and to rebuild American industry. He won because these are good policies and the Democrat policy was to break glass ceilings. The original Trump policies are still good and the Democrats have spent two years examining the entrails of Russian cyber spies. If Trump wanted to implement his policies, he would keep his son-in-law out of the White House, not go within a bull's roar of John Bolton and set up multi-disciplinary committees to produce ideas, programmes and projects after the manner of FDR and the New Deal.
It is possible a new Democrat administration in 2020 led by Elizabeth Warren and Bernie Sanders with a Congress rejuvenated by young talent such as Alexandria Ocasio-Cortez will exercise a civilising influence on American public life but first the Democrats will have to bury their Wall Street, Clintonite faction. Then they will have to deal with the usual suspects – the Anglo-American deep State, the military-industrial complex, the links between rich Saudi-Arabians, rich Americans, rich Zionist Jews, etc.
But this is not about America. It is about China. I would like to think that the Chinese in Xinjiang and Tibet have sown the seeds of their one-Party State's destruction but they play for keeps and these are land-locked provinces. There is not much we can do to help although I did consider sending the Tibetans a copy of The Seven Pillars of Wisdom by T.E. Lawrence when the Chinese were building that mountain railway.
The Lawrence of Arabia movie was faithful to the book but I agreed with my mother's literary criticism. After Lawrence and his team had blown up half a dozen trains it did get boring. The Dalai Lama has a point. Physical resistance to such overwhelming numbers on such remote soil is likely to be counter-productive. The Tibetan Buddhists will have to hope their philosophy outlasts Chinese neo-Leninism. Their position is analogous to that of the Australian Aborigines who are waiting patiently for us whitefellas to go back to England after we have used up all the iron ore and gas.
The island of Taiwan is another matter. I doubt if the Americans will go to war over the South China Sea but they will absolutely go to war over Taiwan. They will have to. If they fail to support Taiwan they will die of shame and Americans do not like to die of shame. Australia will support America in a Taiwan war.
Thus, the hope for peace in our time rests on the ability of Chinese President Xi Jinping and Defence Minister Wei Fenghe to exercise restraint on the Taiwan issue. Toning down their rhetoric would be a good start. Their better policy is to undermine Taiwanese independence by duchessing politicians, businessmen, professors and suchlike folk as they do in Australia, although I can't imagine the Taiwanese would let them get away with such a blatant exercise as The Australian Council for the Promotion of Peaceful Reunification of China. That little Aussie beauty reminds me of the Taliban's Ministry for the Propagation of Virtue and the Prevention of Vice (financed by Saudi Arabia), Senator McCarthy's House Committee on Un-American activities and my all-time favourite propagandist outfit, Soviet Russia's League of Militant Godless.
The Orwellian thought control brought to new heights by the Communist Chinese is unforgiveable, although I dare say we have political and corporate types who wish they could emulate the Politburo's power in Australia, where we leave it to commercial television and professional sporting leagues to turn us all into morons.
There must still be an element of risk in the South China Sea where both sides are batting like millionaires. A Gulf of Tonkin incident would not be difficult to engineer. ABC television's Sunday night serial "Pine Gap" is exploring these themes. This Tinker, Tailor, Soldier, Spy tale for the age of the internet and mobile phones is a neat snapshot of life in the north. Australians and Americans are spying on each other. The Chinese are playing games with both of us and an educated young woman of Aboriginal descent is trying to screw as much money as she can from native title deals. Such is life in outback Oz.
Jerry Roberts was born in the American mid-west and grew up there in the first decade of the Cold War. He has a clear childhood memory of a newspaper on the kitchen table with a picture of Russian tanks rolling into Budapest to crush the 1956 Hungarian uprising. As today's cold war between China and America heats up, he finds himself supporting his native land despite its sins in the Middle East, Central America and elsewhere.
Hi Rosemary and Kim. My post was the least emotional of the war stories published on Pearls and Irritations but that doesn't mean I am a cool customer. I was looking at the geo-politics. My mother used to say that conscription should start at age 60. If G.W. Bush and his little mates, John Howard and Tony Blair, had been called up for front-line infantry they might have found another way of dealing with the axis of evil.
The Chinese will never give up on Taiwan. As with Tibet and Xinjiang it is not about economics and national security. It is about pride. If they take over Taiwan by force and the Americans let them the USA would by definition give up top dog status and I doubt if the Yanks are ready to do that.
The South China Sea islands are curious. The best explanation I can find is that the Chinese are using this build-up to make diplomatic trouble in the neighbourhood — Australia, Japan, the Philippines, Malaysia, Indonesia, Vietnam etc.
Both sides are playing a long game. China may have problems of over-extension. They are duchessing politicians and businessmen from Africa to Australia. The Americans are trying to inflict economic pain under Trump's leadership, probing for Chinese weakness. Australian intelligence agencies are keeping an eye on our politicians and business compradors. American admirals with itchy trigger fingers are looking for an excuse to sink the Chinese fleet. Christopher Clark's book is recommended..
Thanks Jerry – much food for thought there. Agree with your assessment of "Seven Pillars of Wisdom" – although his eye for detail and ability to recollect was astonishing. If you haven't come across it, I can highly recommend "Bell of the Desert" by Alan Gold, a dramatisation of Gertrude Bell's amazing life – an excellent read (or listen as I did).
And on a philosophical note – will we ever see a time when men in power will stop seeing the meek and the powerless as pawns in their games?
…and then there was General Jack Lavalle…scapegoat for a rotten administration? Remember the Armistice – Naturally. And then : don't Forget My Lai, and how the monsters make the rest of us – Pay?


The joy of parenthood is one of the most amazing feelings there is. From the moment a child is born, you already start thinking of their future and how you can help to make it great.
On Children's Day, parents and guardians show their kids just how much they care for them. While buying them the latest toys and gadgets sounds great, why not think further ahead and buy a gift they will treasure forever.
No matter who your child grows up to be, understanding how to manage money will play a big role in their lives. Therefore, it is important you start showing them the ropes early on. Start by teaching them where money comes from, how to keep it safe and how to use it wisely. They will always remember how it feels to work towards a goal and how you helped them get there.
As a parent, it is always wise to think about the future. Setting up an investment for your child is guaranteed to give them an edge in the future. One great way to do this is by gifting them a mutual fund from FBNQuest Asset Management. Monthly contributions can make all the difference and open up a window of opportunity for your child when they need it most.
Starting from as low as N5,000, you can start investing in the future of your child through an FBNQuest Mutual Fund. Get started here: Gift a Mutual Fund.
Oluwatosin Olaseinde: Do You Have an Emergency Fund?


We are pleased to announce the GNU Automake 1.15 minor release.
The most important and subtle of those bugs is bug#18286: "make distcheck"
only corner-cases should be affected -- but better safe than sorry.
instead (which was introduced in Automake 1.13).
disable and enable shell globbing.
DESTINATION argument must be a directory.
associated log file (automake bug#11814).
is more portable and has feature parity with the perl implementation.
'$(top_srcdir)') are also changed accordingly.
reserve the right to change them at any time.
newline (bug#16841).  Regression introduced in Automake 1.14.
automake bug#15981). Bug introduced in Automake 1.14.
dependency tracking and having a ton of source files (bug#18744).
hash keys order in Perl 5.18.
message like "... overrides Automake target '$(srcdir)/foo.am".
he could already do with the .MAKE .and .PHONY targets.
messages (automake bug#16827 and bug#16997).


text:
Rajaram Nityananda (born 1948) is an Indian physicist who works on Solid State Physics, Liquid Crystals, Astronomical Optics, Image Processing, & Gravitational Dynamics. He currently works as professor at Azim Premji University Bengaluru. He was formerly the Director of the National Centre for Radio Astrophysics and also Tata Institute of Fundamental Research (TIFR) Centre for Interdisciplinary Sciences in Hyderabad. He served on the Physical Sciences jury for the Infosys Prize from 2015 to 2017. He also serves as an Associate Editor of the Journal of Astrophysics & Astronomy, published by the Indian Academy of Sciences. He also serves as the chief editor for Resonance Journal for Science Education published by Indian Academy of Sciences. He is also the Chairman of the Board of Directors of the National Institute of Technology, Tiruchirappalli and Chennai Mathematical Institute. He is currently serving as a faculty member at the Indian Institute of Science Education and Research, Pune. Previously he worked at Raman Research Institute from 1975 to 2000.

References

External links
Rajaram Nityananda's articles on INSPIRE-HEP

1948 births
Indian astrophysicists
Living people
Bangalore University alumni

redpajama_set_name: RedPajamaWikipedia


text:
Middle Grade Fiction: Weedflower, by Cynthia Kadohata
Title: Weedflower
Author: Cynthia Kadohata
Publisher's Summary:
Twelve-year-old Sumiko feels her life has been made up of two parts: before Pearl Harbor and after it. The good part and the bad part. Raised on a flower farm in California, Sumiko is used to being the only Japanese girl in her class. Even when the other kids tease her, she always has had her flowers and family to go home to.
That all changes after the horrific events of Pearl Harbor. Other Americans start to suspect that all Japanese people are spies for the emperor, even if, like Sumiko, they were born in the United States! As suspicions grow, Sumiko and her family find themselves being shipped to an internment camp in one of the hottest deserts in the United States. The vivid color of her previous life is gone forever, and now dust storms regularly choke the sky and seep into every crack of the military barrack that is her new "home."
Sumiko soon discovers that the camp is on an Indian reservation and that the Japanese are as unwanted there as they'd been at home. But then she meets a young Mohave boy who might just become her first real friend...if he can ever stop being angry about the fact that the internment camp is on his tribe's land.
This is a well-written piece of historical fiction. Any reader will feel the injustice of the Japanese internment in WWII, as well as the hurt of any child who is rejected for reasons she can't even fully understand. In that, it's a powerful work, and well worth the reading. The book gives the reader a good introduction to what that decision on the part of the US government did to individuals, families, and an entire culture. The author only touches on something that was a major problem, the destruction of the family unit as children banded together and ate apart from their parents or even other siblings, and as families were torn apart by the question of signing the loyalty oath.
The book did feel flawed to me, however, in a couple of ways. One isn't so much a flaw as just how it is put together: the two parts of Sumiko's life almost end up feeling like two separate books, though I think the first part is necessary. The reader needs to see Sumiko doing daily things--not exactly what the reader does, but things that feel human and ordinary and include kid things like sometimes resenting chores--in order to feel fully the shock of the uprooting.
The other issue I had is worse, though I'm not sure how I'd fix it. That is quite simply that the conflict in the story is so huge, and so far beyond Sumiko, that there is no sense really of the development of a story, and certainly no sense of resolution. I don't know; maybe that is just as well if you want to keep the focus on the real injustice. Certainly the characters are well-drawn, so that we care about Sumiko and her life. And I appreciated the way that she, as children do, becomes at home in the camp--at home enough to not want to leave and start yet again. And maybe the fact that she's given no choice about that, either (by the adults in her own family, this time) is no accident: kids are blown by parental winds, in a smaller and more benign echo of what was done to an entire population.
Weedflower is a good book for introducing children to this aspect of WWII. It is suitable for any kids old enough to cope with the frustration at the injustice (maybe 8 and up, depending on the child). I can also recommend books by Yoshiko Uchida, both fiction and her personal narrative (suitable for older children; some libraries put it in adult biographies, some in kids'), and for adults, Looking Like the Enemy, by Mary Matsuda Gruenewald (in the interests of full disclosure, her brother was my Sunday School teacher long, long ago) which tells of one family's experience being deported from the Pacific Northwest to camps in Idaho and Montana.
FTC Disclosure: I checked Weedflower out of my library, and received nothing from the writer or publisher in exchange for my honest review. The opinions expressed are my own and those of no one else. I am disclosing this in accordance with the Federal Trade Commission's 16 CFR, Part 255: "Guides Concerning the Use of Endorsements and Testimonials in Advertising."
The Ninja Librarian and Return to Skunk Corners on sale--just 99 cents for the month of July!
historical fiction Japanese Internment middle grade review WWII


Our signature fruit-only gift basket. The Simply Fruit gift basket is a hand woven wicker basket overflowing with an ambrosial blend of the finest seasonal fruit such as Fuji and Granny Smith Apples; Bosc, Red and Green D'Anjou Pears, and juicy Navel Oranges combined with our Dried Fruit Medley.
1 Dried Fruit Medley - 4 oz.
Agricultual restrictions prohibit fruit trees from being shipped to California and Arizona. In addition, citrus cannot be shipped to Arizona. Northwest-grown pears or apples will be substituted.
Fruits came in good shape and fresh.
The fruit I ordered came in good shape and is being enjoyed.
I sent as a gift for Christmas. The delivery was good and the basket was loved by the receivers. Loved the FREE Shipping.
This was for my 93 year old grandmother and she loved it!
Purchased this basket for Christmas for a family member. He loved it! The basket arrived by the date requested. I'm told the fruit was sweet and delicious. Lovely surprise and thoughtful gift for many occasions.
Always reliable in quality and on-time shipping. Would recommend to others.
The service was impeccable as was the quality of the product. Everything was packed well, arrived promptly , and the quantity of fruit was generous. I'd re-order this item.