Feature Selection Statistical Test for Nominal Response Vs Continuous Predictors? (in R)
I cannot find much information on this, none so-far useful.
I have a sparse data set with 17K+ columns of continuous gene expressions, an example of a typical column: (3.15, 0, 7.1294, 0, 0, 0, 2300.2, 213.444,...) and each row has an associated nominal response of celltype = 'A', 'B', 'C', 'D', 'E'. These are names, hence nominal, and not ordinal.
Thus, there are 17308 gene expression columns, and a single response column in my dataframe, with 4954 rows/samples.
I am trying to find the right statistical test to perform a filter-method for feature selection (taking the best 1000 low-correlation p-values obtained through some statistical test). My teacher mentioned Wilcoxon rank sum test, but I dont see how this applies, nor ANOVA. I found the Kruskal-Wallis as the only relatively good test. I do not come from a statistics background, having only taking Mathematical Statistics I in university.
I use a File Backed Matrix (FBM from bigstatsr package) to assign the p-values in a foreach loop, as follows:
foreach(i = 1:num_feats, .combine = 'c') %dopar% { dat_total = cbind(df[, i], df$ctype) p_vals[i - 1, 1] <- kruskal.test(df[, i] ~ df$ctype, data = dat_total)$p.value } but this results in about 12000 p_values stacked up at zero... (?)
I also tried switching df[, i] ~ df$ctype for df$ctype ~ df[, i] but kruskal.test in R asks for response ~ group, regardless, when I do this I get a bimodal distribution of p-values: a few thousand stacked up near 0 (but non are zero) and a few thousand stacked up near 1 (some equal to 1). I read more on this as in 'Section C' found here.
I'm supposed to just test each column against the response, check for significance, peel off the best 1000, and then generate a feature reduced matrix df_sml that I then feed into elastic net, lasso, linear discriminant analysis and a classification tree. 655
What I did however was pick the best <PHONE_NUMBER> p-values, and pick the top 1000 that had correlation between -0.25 and 0.25 (LDA was failing otherwise).
ANY tips are appreciated, I don't see the point of deriving a test data set for reproduction. This(and several other) articles mentioned Kruskal-Wallis for categorical dependent versus continuous independent variables (but my concern is I am feeding them in 'backwards' to the test in R).
I dont understand why I would possibly want to factor celltype into a dummy variable, as this is nominal data and not ordinal data, so any explanation regarding that appreciated. If it is okay, why? Doesn't ordinal data imply rank? And if I am treating the dummy variable as factor, how is that different than treating 'A', ..., 'E' as factors?
The simplest method you can suggest would be most appreciated, hence my turn to Kruskal-Wallis and not some of the more complicated and involved methods I have seen suggested (I likely wouldn't have the time to implement them).
edit: I wanted to add a few things: I ran elastic net for alpha = 0.1:0.9 by 0.2 and LASSO, and I am getting a classification accuracy rate around 0.98 for all of these. So unless something is amiss, my models are getting pretty good results, LDA performed just slightly worse (0.96) than Elastic Net (glmnet), the Tree model with pruning (0.76) was garbage lol. And this was using df$ctype ~ df[, i] which is group ~ response, not the desired format by kruskal.test as far as I know. {see ?kruskal.test(formula,...)}
edit2: I wanted to add, there are like 8K+ columns with 95% entries being zero, but the teacher said to leave them in and do this test instead (this data is for a machine learning course midterm project), which I've read can be what is contributing to the wild number of p-values stacking up at or near 1. I originally stripped these columns from the dataframe. 458