Correct statistical test to answer: "did my intervention help my patients?" I am doing something to a set of $N$ patients (keeping this vague for generality).
In order to assess if my intervention/treatment is making any difference I did the following:
Before my intervention I measured something about the patients (call that measurement $X_i$ for patient $i$), and repeated the same measurement after my intervention ($Y_i$). So I have $N$ pairs $(X_i, Y_i)$
Let's say my measurements/test spits out a number from 0 to 5 for each patient. Assume I only know high numbers are good and low numbers are bad. (e.g. the difference between a 5 and a 4, might not be necessarily the same as the difference between the 2 and a 1). (I think this is what is called an ordinal measurement). Furthermore assume that when looking at the results of my test for the whole set of $N$ patients we observe that they don't look normally distributed.. (e.g. their most common result could be a 1 while the least common could be a 5)
What is the correct statistical test to see if we could say that my intervention is improving my patients score on those measurements?
I'm also interested in understanding how slight changes to my problem would affect the choice of a statistical test
Does this depend on $N$?
What if actually my measurements are a continuous variable that actually looks normally distributed?
I'm also curious to know if I had first divided the group into 2 groups and only did my intervention to one of the groups, (while for the other group I also measured twice but without any intervention), if this would have been better somehow, and in this case if the test would be different
Thank you

This question is conflating two distinct issues, which both are at the core of statistics: 1) Is this difference just due to sampling error and 2) what does this difference mean?
You seem to be mostly focused on the first one, but the second one is the important one if your real question is "how can I help patients?"
In general, a statistical test is a procedure that is designed to tell you how confident you can be that a difference (e.g. the difference between two groups on some outcome, or the difference between a measurement before and after a treatment)
observed within a random *sample actually reflects an underlying difference in the population from which the sample is drawn, as opposed to being just due to "random chance." That is, a statistical test is a way to guard against sampling error that occurs when you try to use random samples to generalize about populations. And the answer almost always depends on "N" the number of people in your sample. The more people in your sample the less likely it is that a given difference you see is just due to sampling error. In your case a paired sample t test - which tests whether the averages of two measurements from the same sample are different or not, is probably fine. Even though the underlying measurement is ordinal, the average of those measurements is continuous.
However, none of what I just said tells you - by itself - whether the treatment actually helped patients. Whether a treatment works or not is a question of causal inference. And while statistical tests can tell you if a difference is real or not, they can't (by themselves) tell you anything about why that difference existed. To answer that question you have to think about research design. In your case, you are describing what's called a pre-post, no-control group design. You did a measurement of a single group, gave them a treatment, and then measured them again. The problem with this approach is that, even if the group got "significantly" better when you measured them the section time (according to whatever test you ran), that doesn't mean the treatment helped. It might be that they just got better on their own - and would have done just as well if you didn't give them the treatment! No statistical test can solve this problem. You need a different research design.
And indeed what you proposed at the end of your answer is one way to (maybe) solve this problem: "First divided the group into 2 groups and only did my intervention to one of the groups, (while for the other group I also measured twice but without any intervention)." This is what is called a difference-in-differences design. The assumption here (which might be wrong!) is that the chance you observe among the group that didn't get the treatment is what would have happened to the treatment group if they didn't get the treatment. So to estimate the effect of the treatment, you compare the change observed among the control group to the change observed among the treatment groups, the "difference between those differences" is the estimated effect of the treatment. Of course you still have to test if that difference is big enough that it's unlikely to be due to sampling error. In practice people often analyze "Diff-in-Diff" designs with regression models and interaction terms (which you should read up on, since these are critical to the question of causal inference), but the actual "test" that is used at the end of the process is often just a different form of the "t test" used above.
That probably wasn't a very satisfying answer, but your question really gets at the two most fundamental challenges in statistics. Picking the right statistical test for the job is usually not the hardest part of research. It often depends on the measurement level of the variable and a few other things, but you can always figure it out with some googling, and often it doesn't even matter so much if you choose the wrong one. But causal inference is the core of using statistics to actually figure stuff out in the world - and doing it well or poorly is what separates good research from bad.
