Error bars on error bars? Inspired by my recent attendance at an environmental toxicology conference, I have the following question about error bars:
Let's say that I'm drawing samples from some unknown distribution, with finite mean and variance. I want to present the sample mean, and add some error bars. Since I don't know much about the underlying distribution, I just add error bars showing +/- the standard devaition of the samples.
My question is, is there any way I could meaningfully indicate how certain I am of those error bars? Adding error bars to the error bars, so to speak.
As as example, I have drawn 5 samples from some distribution, and I have repeated this 5 times. The sample means, and error bars of +/- the sample standard deviations, are shown below.

We can see that by chance, these sample means and error bars look quite different, and not really mutually compatible. Of course 5 samples isn't very much, but if my samples are obtained via some convoluted experimental procedure (capturing a wild animal and taking a blood sample, for example), it might not be an easy option to get more samples.
Update:
Just to add some notes on how I was thinking:
Coming from a computational physics background myself, I'm used to Monte Carlo methods, and the $1/\sqrt{N}$-error which follows from the central limit theorem. So just like the error in the sample mean has an expected distribution, I thought perhaps it would make sense to ask about the expected error in the sample standard deviation. Of course, the problem is that the distribution of the error in the sample mean is expressed in terms of the (unknown) variance of the underlying distribution, and hence I am left taking the standard deviation of the sample, or something along those lines.
But still, I thought there ought to be some way of indicating that my sample standard deviation is itself quite uncertain, due to the small $N$. But perhaps the only way is simply to list $N$, and be explicit about what the error bars show.

Review of confidence intervals
Let $\theta \in \mathbb{R}$ be a parameter of interest which we study based on a random variable $X$. An exact $1-\alpha$ confidence interval $(L(X),U(X)$ is defined by the property that
\begin{equation*}
 \mathbb{P}\left[ L(X) < \theta < U(X) \right] = 1-\alpha,
\end{equation*}
where $L$ is the lower endpoint and $R$ is the upper endpoint of the confidence interval.
The plot shown in the question illustrates that $L$ and $U$ are random variables. This is certainly the case, as they depend on the random variable $X$. However, a fraction of the confidence intervals $(L(X),U(X))$ contain $\theta$. By construction, the fraction is exactly $1-\alpha$. When $\alpha=0.05$, this is $95\%$ of the confidence intervals.
Error bars on error bars
This procedure makes perfect sense if the target of inference is $\theta$ - which is what we stated above. However, you may also be interested in the endpoints $L(X)$ and $U(X)$ themselves. Then you can construct a "confidence intervals" $(L^L(X), U^L(X))$ and $(L^U(X), U^U(X))$ such that
\begin{equation*}
 \mathbb{P} \left[L^L(X) < L(X) < U^L(X) \right] = 1-\alpha
\end{equation*}
and
\begin{equation*}
 \mathbb{P} \left[L^U(X) < U(X) < U^U(X) \right] = 1-\alpha.
\end{equation*}
For example, the "confidence interval" $(L^L(X), U^L(X))$ contains the random variable $L(X)$ a fraction $1-\alpha$ of the time.
Based on all these confidence intervals, we could extend the original confidence interval to $(L^L(X), U^U(X))$. I'm not sure what the utility of this is, though.
