Correct statistical test to answer: "did my intervention help my patients?" I am doing something to a set of $N$ patients (keeping this vague for generality).
In order to assess if my intervention/treatment is making any difference I did the following:
Before my intervention I measured something about the patients (call that measurement $X_i$ for patient $i$), and repeated the same measurement after my intervention ($Y_i$). So I have $N$ pairs $(X_i, Y_i)$
Let's say my measurements/test spits out a number from 0 to 5 for each patient. Assume I only know high numbers are good and low numbers are bad. (e.g. the difference between a 5 and a 4, might not be necessarily the same as the difference between the 2 and a 1). (I think this is what is called an ordinal measurement). Furthermore assume that when looking at the results of my test for the whole set of $N$ patients we observe that they don't look normally distributed.. (e.g. their most common result could be a 1 while the least common could be a 5)
What is the correct statistical test to see if we could say that my intervention is improving my patients score on those measurements?
I'm also interested in understanding how slight changes to my problem would affect the choice of a statistical test
Does this depend on $N$?
What if actually my measurements are a continuous variable that actually looks normally distributed?
I'm also curious to know if I had first divided the group into 2 groups and only did my intervention to one of the groups, (while for the other group I also measured twice but without any intervention), if this would have been better somehow, and in this case if the test would be different
Thank you

Fix your experimental design first; then use an ordinal regression model
Firstly, before you get to the inference phase, you need to alter your experimental design.  Since you are trying to infer the causal effect of a medical treatment, you should use randomised control and treatment groups (e.g., using block randomisation) to allow you to infer the causal effect from a comparison of these groups.  Without a control group, you have no way of knowing if a statsitical change in the outcome occurred because of your treatment or because of some contemporaneous change in the patient cohort from another source.  Depending on the natue of the medical treatment, you should also consider a "blinding" protocol to prevent the patients from knowing which group they are in, and possibly even a "double blinding" protocol to also prevent the treating doctor from knowing which group each patient is in.  Randomisation and blinding are protocols used to sever the statistical connection between the treatment variable and any possible confounding variables in the analysis.
Once you have fixed your experimental design, you will then have data that can be used to infer a causal effect.  It sounds like you are using a five-point ordinal scale for your outcome variable, so you would model this using some kind of ordinal regression model.  Typically, you would form a regression model that includes the treatment variable, relevant demographic variables, and any other useful variables relating to the medical state of the patients.  (Make sure to exclude any mediator variables --- i.e., variables that are intermediate medical outcomes from the treatment to a change in the outcome.)  You would then use this model to infer the effect of the treatment variable on the ordinal outcome variable when other (non-mediating) variables are held constant.  In answer to your question, yes, the strength of the inference will depend on the number of data points in your analysis --- roughly speaking, the more patients you have in your sample the more accurate your inferences about the effect of the treatment.
If you will permit me to speculate a little, it appears from your question that you are probably a novice in the field of medical experimental design.  In almost all institutions that perform medical research (universities, hospitals, etc.) there are consulting statisticians available to give assistance on experimental design, modelling and inference.  Even if you are outside these institutions (e.g., if you are a GP in private practice) you will probably find that consulting statisticians in these institutions will be happy to help you with some basic advice.  I strongly recommend going to talk to a consulting statistician working in the medical/population health sector for assistance with planning and executing your medical experiment.  Medical experimentation generally also requires ethical clearance from relevant bodies and so you should ensure you have gone through an appropriate ethics process for your study prior to executing it.
