In ARMA models, Is the withe noise "correlated" with the process? I know that an ARMA(p, q) refers to the model with p autoregressive terms and q moving-average terms. For example:
$$ X_t = c + \varepsilon_t +  \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i}  $$
where $\varepsilon_{t} \sim WN(0,\sigma^2)$. That is, $E(\varepsilon_t) = 0$ and $E(\varepsilon_t \varepsilon_{t-j}) = 0$, for $j\neq 0$. Equivalently, $E(\varepsilon_t | \varepsilon_{t-j}) = 0$.
But in the books I've read, it doesn't tell me anything about the relationship between all the lags of $\varepsilon_t$ and all the lags of $X_t$. For example:

*

*Are $X_t$ and $\varepsilon_t$ independent? And how about $X_t$ and $\varepsilon_{t-j}$?

*If not, are they mean independent? That is: $E[X_t | \varepsilon_{t-j}] = X_t$?

*Are they not correlated? That is, $cov(X_t,\varepsilon_{t-j})=0$?

Some help?

*

*No, $X_t$ and $\varepsilon_t$ are dependent. $X_t$ is a linear function of $\varepsilon_t$, and $X_t$ is positively correlated with $\varepsilon_t$: \begin{aligned}
\text{Cov}(X_t,\varepsilon_t) &= \text{Cov}([c + \sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i}] + \varepsilon_t,\varepsilon_t) \\
&= \text{Cov}(\sum_{i=1}^p \varphi_i X_{t-i} + \sum_{i=1}^q \theta_i \varepsilon_{t-i},\varepsilon_t) + \text{Cov}(\varepsilon_t,\varepsilon_t) \\
&= 0 + \sigma^2 \\
&> 0.
\end{aligned}
$X_t$ and $\varepsilon_{t-j}$ are also dependent. Rewrite ARMA as MA($\infty$), and you will find that $X_t$ is a linear function of $\varepsilon_{t-j}$, and thus an analogous argument to the one above holds also here.

*No, because of 1.

*No, because of 1.

If you prefer empirical results to analytical proofs, simulate an ARMA process and check the empirical cross-correlation function of $X_t$ vs. $\varepsilon_t$. Here is how to do that in R:
n=1e6 # length of time series
set.seed(1)
eps=rnorm(n)
x=arima.sim(n=n, list(ar=c(0.8897,-0.4858), ma=c(-0.2279, 0.2488)), sd=1, innov=eps)
ccf(x,eps)


