Table 1. Summary table of the various regression methods explored in this thesis Method OLS PLS LASSO ELASTIC NET SPLS Shrinks some Generalizes the lasso. Shrinks coefficients and sets some coefficients and sets Chooses subspaces from others equal to zero in Adds an to L1-constraint others equal to zero; averages explanatory matrix, X, sequentially accordance with the PLS algorithm to highly correlated features and projects response vector, y, onto the shrinkage parameter, impose sparsity. t. shrinks averages. Provides a Summary subspaces of the column spaces of Provides a sparse model Performs feature X sparse model that has more to determine the model variable that can be used for selection yet retains terms than the lasso and can coefficients. both feature selection stable properties of PLS. be used for feature selection and composition and composition predictions. predictions. , None Penalty Tuning None # of dimensions k, t t, α κ, t1, t2, K Parameters Provides an interpretable model, selects subset of Performs well in the >> p N predictors with the case. Provides an interpretable Performs well in the >> p Advantage(s) Provides a stable model. strongest effects on the model that is more stable than case. N response variable. Can the lasso. Useful for feature be used for feature selection. selection when less data are available. Does not provide a Does not perform as stable model Provides a complex model in which well in the >> case. p N when the Cannot be used for feature all coefficients are linear Arbitrarily chooses one variables are selection in situations when combinations of the original covariate from a group Shrinkage properties not Disadvantage(s) correlated; less data are available because channels. Involves a complex of highly collinear yet well understood. predictions it overwhelms the data with optimization problem with no covariates to use in the suffer from too many model variables. simple, closed-form representation. model and discards the wild variance rest. so they are unreliable.
Table 1. Summary table of the various regression methods explored in this thesis Method OLS PLS LASSO ELASTIC NET SPLS Summary Chooses subspaces from explanatory matrix, X, sequentially projects response vector, y, onto the subspaces of the column spaces of X to determine the model variable coefficients. Shrinks some coefficients and sets others equal to zero in accordance with shrinkage parameter, t. Provides a sparse model that can be used for both feature selection and composition predictions. Generalizes the lasso. Shrinks some coefficients and sets others equal to zero; averages highly correlated features and shrinks averages. Provides a sparse model that has more terms than the lasso and can be used for feature selection and composition predictions. Adds an L1-constraint to the PLS algorithm to impose sparsity. Performs feature selection yet retains stable properties of PLS. Penalty None , Tuning Parameters None k, # of dimensions t t, α κ, t1, t2, K Advantage(s) Provides a stable model. Provides an interpretable model, selects subset of predictors with the strongest effects on the response variable. Can be used for feature selection when less data are available. Performs well in the p >> N case. Provides an interpretable model that is more stable than the lasso. Useful for feature selection. Performs well in the p >> N case. Disadvantage(s) Does not provide a stable model when the variables are correlated; predictions suffer from wild variance so they are unreliable. Provides a complex model in which all coefficients are linear combinations of the original channels. Involves a complex optimization problem with no simple, closed-form representation. Does not perform as well in the p >> N case. Arbitrarily chooses one covariate from a group of highly collinear covariates to use in the model and discards the rest. Cannot be used for feature selection in situations when less data are available because it overwhelms the data with too many model variables. Shrinkage properties not yet well understood.