When should I use the delta method rather than the parametric bootstrap? Suppose I am willing to assume a particular likelihood function for an applied statistics problem. I am able to derive the MLE for the parameter $\theta$, which I will call $\hat{\theta}$. I can also derive a standard error for $\hat{\theta}$.
I am also interested in generating a point estimate and confidence interval for some transformed parameter $\tau = g(\theta)$. A statistics textbook might remind me that $\hat{\tau} = g(\hat{\theta})$ is the MLE for $\tau$ due to the equivariant property of the MLE and that I can use the delta method to get standard errors for $\tau$.
I can also use the parametric bootstrap to get confidence intervals for $\hat{\tau}$ (and, for that matter, $\hat{\theta}$), since I am assuming a parametric model.
I have a suspicion that the parametric bootstrap is a "safer" way to form confidence intervals than the delta method, but that the delta method is emphasized in statistics classes because it is a nifty mathematical result and because of historical reasons (for example, statistical instruction predates fast computers). Downsides to the delta method are:

*

*CIs based on a normal approximation often don't work because asymptotic normality hasn't "kicked in" yet

*It may be hard to simply time consuming to derive

*Even when it is tractable, it's easy to make math mistakes in deriving the standard error

My question is: in practical applications (rather than when we are doing school or theory), do people really find the delta method more useful than the parametric bootstrap in many cases? Or should I think of the parametric bootstrap as my "default" method for computing CIs in this situation (with the possibility of overruling this default, if there are special circumstances to the problem at hand)? Or perhaps these questions are wrong-headed for some reason I haven't considered?

You will often be disappointed in the confidence interval coverage probability in at least one of the two tails when using either the bootstrap or the delta method.  This is one area where traditional statistical education, which focuses on frequentist methods, has done us a disservice.  Contrast this with Bayesian modeling and using MCMC to generate thousands of samples from the multivariate posterior distribution of all the parameters.  Given these samples, one merely transforms the parameters to any desired new parameter, transforming all of the posterior samples into the new metric, then computes things such as highest posterior density uncertainty interval.
The delta method assumes that the sampling distribution of the derived parameter is approximately symmetric, which is very often not the case.  The Bayesian solution is not only easy to program and doesn't assume that anything is normal (or even symmetric) but provides exact inference on the derived parameter to within simulation error.
