How does the Bayesian Information Criterion work for model selection? I am aware that we can use the BIC values from different models in order to determine which model predicts the data best. However, I'm a little confused about the criteria used to determine which model is better over another model.
For the BIC, a difference of 2 is considered as positive evidence for one model over the other and a difference greater than 10 suggests very strong evidence in favor of one model over the other.
Let's assume one model has a BIC of 7461 and another has a BIC of 7508. The two models differ by 47. Therefore, we would conclude that the model with BIC 7461 explains the data the best. Below are my questions about this conclusion/interpretation:

*

*What are the units of the BIC numbers?

*Do we have to consider the proportion of difference between the two models in our interpretation?

*Does a difference of 2 always indicate positive evidence for one model over the other regardless of the BIC values? For example, a BIC(1) = 10 vs BIC(2) = 12 has a difference of 2 and proportionally BIC(2) is 1.2 (12/10) times greater where as BIC(1) = 7461 and BIC(2) = 7508 differ by 47 but proportionally BIC(2) is 1.006 (7508/7461) times greater.

*Do the units matter with BIC? Or, is it always the case that a difference of 2 means positive evidence regardless of how large or small the BIC values are?



For the BIC, a difference of 2 is considered as positive evidence for one model over the other and a difference greater than 10 suggests very strong evidence in favour of one model over the other.

Where did you get this from? I don't think it makes any sense.

Re: your other questions:

What are the units of the BIC numbers?

Formally, BIC is defined as follows:
$$
\operatorname{BIC}=-2\ln\hat{L}+p\ln(n)
$$
where $\ln\hat{L}$ is the maximised value of the log-likelihood; $n$ is the number of observations; and $p$ is the number of parameters in the model.
The smaller the BIC, the better. This is because a higher maximised log-likelihood will lead to a smaller BIC. Note that BIC is penalised by the number of parameters that need to be estimated. As such, BIC tends to favour simple (more parsimonious) models.

Do we have to consider the proportion of difference between the two models in our interpretation?

Not really. It sounds like you are worried about whether the difference between two models' BICs is statistically significant or not. In my experience, bootstrapping is the best way to do this.

Does a difference of 2 always indicate positive evidence for one model over the other regardless of the BIC values?

No.

Do the units matter with BIC? Or, is it always the case that a difference of 2 means positive evidence regardless of how large or small the BIC values are?

No; and definitely not.
