Predictive distribution in the Frequentist setting Let $\{X_t\}_{t \in T}$ be a time series, such that $X \sim F(\theta)$, for some arbitrary distribution $F$. Based on the observed values $\{x_1, x_2, \cdots, x_{t-1}\}$, suppose that I want to predict $x_t$.
What should I do? My suggestion would be: estimate $\theta$, say, by $\hat{\theta}$ (e.g., MLE), $2)$ simulate sufficiently many points from $F(\hat{\theta})$, and $3)$ average values from step "$2)$" in order to obtain $\hat{x}_{t}$.
However, the above procedure has two problems (maybe more): $A)$ I am not accounting for the uncertainty in estimating $\theta$, and $B)$ I am obtaining a "deterministic point prediction", as opposed to a "probabilistic forecast" (in the form of a predictive distribution, which is what I want).
Assuming $X$ is absolutely continuous and, therefore, has a pdf $f_X$, I think I could address item $B)$ by arguing that $f_X(x|\hat{\theta})$ describes the predictive distribution. Is it correct? Even if it is true, I still have the problem described on item $A)$.
Also, under this same framework, how could I predict $\{x_{t+1}, x_{t+2}, \cdots, x_{t + p}\}$ based on the same original data set? In this case, the uncertainty must increase with $t$, right?

I think my question can be summarized by: "under the frequentist framework, how to make 'probabilistic forecasting' appropriately?".

The principle of confidence intervals can be turned into a confidence distribution or fiducial distribution.
In the same way, you could turn prediction intervals into a prediction distribution.
