Error bars on error bars? Inspired by my recent attendance at an environmental toxicology conference, I have the following question about error bars:
Let's say that I'm drawing samples from some unknown distribution, with finite mean and variance. I want to present the sample mean, and add some error bars. Since I don't know much about the underlying distribution, I just add error bars showing +/- the standard devaition of the samples.
My question is, is there any way I could meaningfully indicate how certain I am of those error bars? Adding error bars to the error bars, so to speak.
As as example, I have drawn 5 samples from some distribution, and I have repeated this 5 times. The sample means, and error bars of +/- the sample standard deviations, are shown below.

We can see that by chance, these sample means and error bars look quite different, and not really mutually compatible. Of course 5 samples isn't very much, but if my samples are obtained via some convoluted experimental procedure (capturing a wild animal and taking a blood sample, for example), it might not be an easy option to get more samples.
Update:
Just to add some notes on how I was thinking:
Coming from a computational physics background myself, I'm used to Monte Carlo methods, and the $1/\sqrt{N}$-error which follows from the central limit theorem. So just like the error in the sample mean has an expected distribution, I thought perhaps it would make sense to ask about the expected error in the sample standard deviation. Of course, the problem is that the distribution of the error in the sample mean is expressed in terms of the (unknown) variance of the underlying distribution, and hence I am left taking the standard deviation of the sample, or something along those lines.
But still, I thought there ought to be some way of indicating that my sample standard deviation is itself quite uncertain, due to the small $N$. But perhaps the only way is simply to list $N$, and be explicit about what the error bars show.

The short answer is "no."
However you construct your error bars, they are a rule.  You cannot be unsure of them.  Let us imagine that they are confidence intervals.  There are multiple standard ways to create confidence intervals.  They are different rules with slightly different properties.  However, they are a chosen rule.
Other ways to construct error bars exist as well, such as adding plus or minus one standard deviation.  It is still a rule.
You know the answer exactly.  They are not uncertain.
What they are reflecting is the random elements of the samples seen.  If they are a $1-\alpha$ percent confidence interval, there is a guarantee that the confidence intervals cover the parameter at least $1-\alpha$ percent of the time.  There is no guarantee that it covers it for this sample.  Even with a set of five samples, none of them may cover the parameter, the guarantee is over infinite repetition.
Each way you could construct an error bar has some form of optimality principle behind it.  So, error bars satisfy some optimality condition that is good on average.
All of them are a statement of the best estimator of the range in which a parameter sits, given a model and a loss function.
Your error bars are a statement of uncertainty.
