Can "Heavily Skewed" Predictor Variables be "Useful" to a Model? In school, we were always told some general/informal "heuristics" about statistical modelling. In particular, we were advised about the following point:

*

*If most of the variable contains zeros (or the same value), it likely will not contribute any predictive power to your statistical model, and it is better to omit this variable from the model.

I was wondering if this heuristic is well supported, or if this is simply a case-by-case thing.  To demonstrate this, I generated the following (comical) dataset - Given the height, weight, gender and if someone owns a yatch, can we predict if this person is a billionaire?
#first dataset
id = 1:100
height = abs(rnorm(100, 170,10))
weight = abs(rnorm(100,90,10))
gender <- c("male","female")
gender <- sample(gender , 100, replace=TRUE, prob=c(0.5, 0.5))
is_billionaire <- c("yes","no")
is_billionaire <- sample(is_billionaire , 100, replace=TRUE, prob=c(0.01, 0.99))
owns_yatch <- c("yes","no")
owns_yatch <<- sample(owns_yatch , 100, replace=TRUE, prob=c(0.01, 0.99))
first_dataset = data.frame(id, gender, height, weight, owns_yatch, is_billionaire)

# second dataset
id = 101:110
height = abs(rnorm(10, 170,10))
weight = abs(rnorm(10,90,10))
gender <- c("male","female")
gender <- sample(gender , 10, replace=TRUE, prob=c(0.5, 0.5))
is_billionaire <- c("yes","no")
is_billionaire <- sample(is_billionaire , 10, replace=TRUE, prob=c(0.90, 0.1))
owns_yatch <- c("yes","no")
owns_yatch <<- sample(owns_yatch , 10, replace=TRUE, prob=c(0.90, 0.10))
second_dataset = data.frame(id, gender, height, weight, owns_yatch, is_billionaire)

#final dataset
final_dataset = rbind(first_dataset, second_dataset)

The dataset looks something like this:
  id gender   height    weight owns_yatch is_billionaire
1  1 female 177.7575  92.28024         no             no
2  2 female 162.6311  87.18358         no             no
3  3   male 175.7431  82.81846         no             no
4  4 female 169.9650  95.32557         no             no
5  5 female 175.7480 100.42995         no             no
6  6 female 170.3356  90.71610         no             no

If we were to run some exploratory data analysis, we would clearly see the strong relationship between owning a yatch and being a billionaire:
library(GGally)


ggplot <- function(...) 
    ggplot2::ggplot(...) + scale_color_brewer(palette="Purples") + scale_fill_brewer(palette="Purples")
unlockBinding("ggplot",parent.env(asNamespace("GGally")))
assign("ggplot",ggplot,parent.env(asNamespace("GGally")))

graph_corr <- ggpairs(final_dataset[,-1], title = "The Effect of Different Variables on Being a Billionaire", mapping = aes(color = is_billionaire)) 
graph_corr <- graph_corr + theme_minimal()
graph_corr


My Question: In this case, we can see the logical and strong effect of a "heavily skewed variable" on the response variable. This being said, can someone please comment on this heuristic - can heavily skewed and "uninformative" variables be useful in statistical models (e.g. regression)?
Thanks!

This is bad advice. Of course skewed data ( features) can be (very) helpful for prediction.  Skewed data is arguably more informative in the real world than "nice" normal looking data.
Just naming a few examples

*

*Income (highly skewed) predicts which car you likely buy


*How often you are behind your credit card payments ( lots of 0s in that feature across clients of a bank) is a good predictor of whether you can pay off your mortgage


*Earthquake seismic amplitude (very skewed) predicts casualties / cost of reconstruction
I m sure you can think of dozens of other examples in your daily life.
I assume this "advice" came from a linear model only pov.  And even for those it's not good / generic advice either.
