Why is t-test more appropriate than z-test for non-normal data? (Disclaimer: I am aware that some form of this question has been asked many times on this site, e.g. this, this and this - I was hoping to find a definitive answer to this from Google but I am still confused by reading these links and I don't have enough privilege to comment, so here goes this question.)
Mathematically, it is necessary to assume data is normal, for t-test to be applied. (This is because of the denominator in t-statistic, where we assume sample variance distributes like chi squared after normalization)

*

*The most popular answer to justify "t-test is good enough for non-normal population" is to say that for large sample sizes, the sample mean is approximately normal by Central Limit Theorem. There is no guarantee on how large the sample size is needed, but we can take it on faith.

*I'd argue that by the exact same argument, we can just use z-test instead of t-test, especially since t-distribution is close to normal when n is "big", so Slutsky's Theorem would apply.

*This must mean that if we favor the t-test over the z-test, we must believe that in a transitional range where n is not big enough, the t-test is better than the z-test in some way, even for non-normal data (e.g. the t-test is more powerful than the z-test). Why? What are the assumptions that go into such statement, or is it purely empirical?

Now this isn't going to be a very rigorous answer, but take it for what it is.
I want to distinguish between two things. There is a difference between your data, and the statistics that you compute from your data.
T-tests are for statistics, not the data itself. The statistics will be normally distributed. Plenty of proofs of the CLT that we could refer to. It is this fact that you are leaning on that makes this work. But wait, why are we using t-tests if I said that your statistic is guaranteed to be normally distributed.
The answer is that you are using your statistic to compute a new statistic about itself. This new secondary statistic is the variance of your guaranteed normal distribution. It is an estimate not of the variance of the data, but of the statistic. This is known as a standard error.
Now, you have two sources of error about the distribution for your statistic.

*

*The original sampling error from the population when you compute your statistic

*The additional error from estimating the variance of your statistic utilizing your statistic.

You use a t-test to adjust for the uncertainty coming from source 2 of your error. The bigger your samples are, the less variance you expect will come from source 2, and your sample will approach a z-test, which would only consider errors coming from source 1, which you can't get rid of.
Essentially, doing a t-test is an act of humility where you are admitting that errors may be creeping into your confidence intervals because of the way that you are computing them. There is no way to get rid of these potential errors, so we are going to make them slightly larger than you would see with a z-test. Of course, there are derivations for why the t-distribution is the correct distribution to use to do this and so forth (I won't subject myself and you to the same tortures I did in school with gamma functions but you can look them up if you are brave enough), but that is the intuitive "why".
