Proof of statistical assumptions behind the methods or tests When reading statistics literature, they provide the assumptions of when specific tests/methodologies can be used. Take factor analysis as an example; you may find this inside a paper: "Generally speaking, cases with missing values are deleted to prevent overestimation (Tabachnick & Fidell, 2007). "
I wonder if the assumptions behind statistical methods/tests are all the fruits of experiments in research, so we just rote memorise the rules and use them in industrial applications? In an industrial setting, I witness people do it in this way, but I'm not confident what if they are someday being asked how to prove the assumptions are necessary, will they be able to explain that? As I'm not aware of how exactly all these assumptions and rules can be explained easily if there are any people who challenge these assumptions at all.
But I am a bit shaky in applying something I can't explain very clearly. I find this question dangling around my mind whenever I need to use any statistical techniques: Do we need to know how to prove things from the ground up before we apply methods in statistics? If not, when would it be helpful if I knew how to prove some assumptions behind the method (if they can be proved at all)? How do I balance practicality and proofs in the statistical world? I find those assumptions challenging to remember and not intuitive at all! How to make it more intuitive so I can remember not to miss a single assumption next time I apply them?
Is there any body of knowledge that lays the foundation for all the assumptions for all statistical tests/methods?
TIA.

A lot of older applied statistics and econometrics textbooks start with a wide range of assumptions, without always making them clear. Readers then learn a wide range of tools based on (possibly strong) assumptions, and only much later come back to re-examine the assumptions.
A new school of agnostic statistics practitioners are flipping this in a really exciting way. In their work, some of the it very very new, they offer introduction that starts with no assumptions. They present first principles and then slowly adds assumptions in a way that makes crystal clear what assumptions we are making, why we're making them, and how we might think about assessing whether they are reasonable or not.
Some names in this line of agnostic regression are David Friedman, Winston Lin, and P Aronow.
Refer to any of David A Freedman's books.
For the best recent work, see Aronow and Miller's recent textbook, Foundations of Agnostic Statistics: https://www.amazon.com/Foundations-Agnostic-Statistics-Peter-Aronow-dp-1316631141/dp/1316631141/ref=mt_other?_encoding=UTF8&me=&qid=
