Can an irrelevant variable be significant in a regression model? I'm reflecting about statistical significance and I think a variable can be irrelevant and significant at the same time. But, how do I explain that? And if that were the case, should I include that variable to my regression model?


I think a variable can be irrelevant and significant at the same time. But, how do I explain that?

This can be explained by using the concept of type I errors.
Below is an example by repeating a t-test 1000 times where we test whether the random number generator has a mean different from zero. Say that we consider a p-value below 0.05 significant, the we find 41 times that the mean of the number generator is significantly different from zero.

p = rep(0,1000)
for (i in 1:1000) {
   set.seed(i) 
   p[i] = t.test(rnorm(100,0,1))$p.value
}

plot(p, bg = 1 + (p<0.05), col = 1 + (p<0.05), pch = 21, cex = 0.5,
     main = "p values in 1000 simulations of a t-test with different seed\n 41 cases of significant p<0.05 level", xlab = "seed number", ylab = "p-value")
lines(c(0,1000),c(0.05,0.05), lty =2)

Significant observations may happen even when there is no true effect (the null hypothesis is true) because sampling is subject to random statistical variations.
The point of 'significance' is to tell whether some observation is extreme and express it in terms of a probability, but an extreme/significant observation does not mean that it is not able to occur when there is not a true (relevant) effect. An extreme/significant observation can happen by chance. (Like in the figure above it happened 41 times out of 1000 experiments)
So that is what significant means from a statistical point of view

an extreme observation that falls outside the range of likely statistical variations that might be expected.

Significance does not directly mean a 'relevant' variable.
This is a bit of a language problem as well. Statistical significance relates to probability of observations. It is not 'significance' as in 'important' as it is used in common language.
Significance is also mostly important when it is absent. Significance is more like a minimal condition for some observation/variable to be important, and it is not like a sufficient condition.
