non-overlapping CIs in predicted probabilities after multinomial logit: can we conclude statistically significant difference? I produced the below plot of predicted probabilities after running a multinomial logit model with Stata (full code below). Questions:

*

*In 2013 (for example), the 95% CIs for M, F, and U do not overlap. Can I therefore conclude that the expected probabilities are statistically significantly different (at 5% level) for these 3 groups?


*For the F group the CIs do not overlap for 2013 v 2008. Can I conclude the predicted probabilities are statistically significant (5% level) for those 2 years?
I understand if the CIs do overlap, they may or may not be statistically significantly different.
Code:
mlogit gender_n year, vce(cluster person)
margins, at(year = (2008(1)2013))
marginsplot

Plot:

Note: cross posted to Statalist but no responses.

A similar question (asking sort of the opposite, what if intervals are overlapping) occured here Why is mean ± 2*SEM (95% confidence interval) overlapping, but the p-value is 0.05?
Making comparisons on standard errors or confidence intervals is a rule of thumb using
$$z_{overlap} = \frac{\vert \bar{X}_1- \bar{X}_2 \vert}{SE_1+SE_2} \geq 2$$
whereas the t-value is more accurate
$$t = \frac{\bar{X}_1 - \bar{X}_2}{\sqrt{SE_1^2+SE_2^2}}$$
Typically you get $z_{overlap}<t$ and it underestimates the distance and overestimates the p-value. So even when the intervals are just overlapping, the 'correct' difference (the $t$ value) might be larger and the p-value smaller than 5%. And if the intervals are not overlapping than the p-value will be even smaller.
If the confidence intervals are not overlapping then the $t$ value will be even larger than 2, and the p-value should be below 5%.
Exceptions In the answer to that question two exceptions where mentioned


*

*In the case of the pooled variance, you might have a situation - although it is rare - that the variance of the larger sample is larger than the variance of the smaller sample, and then it is possible that $t<z_{overlap}$.


*Instead of z-values and a z-test you are actually doing (should be doing) a t-test. So it might be that the levels on which you base the confidence intervals for the error bars (like '95% is equivalent to 2 times the standard error') will be different for the t-test. To be fair, to compare apples with apples, you should use the same standard and base the confidence levels for the error bars on a t-test as well. So let's assume that also for the t-test the boundary level that relates to 95% is equal to or less than 2 (this is the case for sample sizes larger than 60).

These are the exceptions, when we can not conclude with certainty that the p-value is less than 5% when 95% confidence intervals are not overlapping.
