Comparing outcomes of two treatment groups: t-test/Mann-Whitney U versus regression I'm a student and had a question about statistical analysis.
I'd like to compare post-treatment outcomes between two groups: group A who received a traditional drug (n = 200), and group B who received a newer drug (n = 100). Some post-treatment outcomes are continuous, whereas others are binary (e.g. % developing organ failure, mortality). The question is: which drug is superior? Group allocation was non-random, i.e. based on doctors' choices/clinical factors - more of an observational study than an RCT.
It has been suggested to me that I should code the drug such that Group A is '0' and group B is '1', and run regression models to evaluate the explanatory power of drug choice on the various outcomes (linear for continuous dependent variables, logistic for binary dependent variables).
My understanding so far is that there are 2 advantages of regression: controlling for confounders, and seeing associations of other variables.
However, I also read that there are 5 assumptions for linear regression models. When I began testing my models for these, I found that my models don't have normal residuals (and so homoscedasticity also becomes hard to test). I don't have the time nor skill to transform / manipulate my data. The other issue is that for some outcomes, there are far fewer data points i.e. n = ~40.
My question is: would a t-test/Mann-Whitney U (depending on normality), comparing mean values for the continuous outcomes between Group A and Group B be okay in this scenario, instead of regression? I've done demographic comparisons and only age differs between the groups; can I report the t-test/MWU P-values with the caveat that any significant results could also be due to differences in age? And for binary outcomes like % organ failure, I could just use 2-tailed tests of proportion?
Would be grateful for some help! I think some versions of this question have been asked before, but I couldn't find a conclusive answer.


Would a t-test/Mann-Whitney U (depending on normality), comparing mean values for the continuous outcomes between Group A and Group B be okay in this scenario, instead of regression?

No, because two-sample comparisons tests don't allow for covariate adjustment. You should do regression analysis instead as argued by @Dave.
You are working with a non-randomized study. And since treatment assignment wasn't randomized you cannot assume that groups A and B are comparable (exchangeable). If there is a difference between post-treatment outcomes between A and B, is it because the new treatment made a difference or because patients given the new treatment were younger/older/sicker/healthier/...? So you have to include covariates in the analysis to adjust for any systematic differences between groups. (And even then you have to assume you've adjusted for all necessary covariates, an unverifiable assumption.)

I've done demographic comparisons and only age differs between the groups.

It sounds like you ran multiple separate tests to check if groups A and B differ in terms of a demographic covariate. However, you don't know the power of the individual tests and more importantly doctors determine treatment by considering all (known) relevant factors together. If you want to check for balance between groups A and B you can estimate propensity scores by fitting a model to predict treatment assignment based on all known covariates.

can I report (...) P-values with the caveat that any significant results could also be due to differences in age?

You need a much stronger caveat than that. Even if you include either the demographic covariates or the propensity scores in the regression, you cannot claim that the analysis is adjusted for all systematic differences between groups A and B.
That's the challenge with non-randomized studies. We can adjust only for covariates that have been measured, while what we really want to do is adjust for all covariates (both latent and measured) that differ systematically between groups. In your study, you want adjust for all factors that doctors took into consideration when they decided whether to assign the traditional treatment or the new treatment.
So in the caveat, you should list all covariates which you adjust for as well as mention any additional factors that are considered clinically relevant.
