When is a model high-dimensional? Usually a model is considered to be high-dimensional when $n \ll p$, where $n$ is the number of the observations and $p$ the number of the variables/features (e.g. Bühlmann and van de Geer, 2011). However, you can find other definitions in the literature, as in Belloni et al. (2018) Ref. They write:

High-dimensional models are characterized by having a number of unknown parameters that is not vanishingly small relative to
the sample size.

So when is a model high-dimensional? Is there a rule of thumb or some criterion, which defines that a model is high-dimensional different than $n \ll p$? I know that there is similar question on StackExchange (Ref), but this post is relatively old and the field of high-dimensional statistics/ machine learning developed in the last ten years.

An excellent introduction to the statistical aspects of this question is Chapter 1.1 in this textbook (I can recommend this book more generally). In a nutshell, a model is called high-dimensional if p (the number of parameters in the model) is of the order of n (the number of observations from that model) or larger. This definition agrees with the definitions already given, but the book makes an important distinction between high-dimensional models and high-dimensional statistics:  "high-dimensional statistics specializes in models that have many parameters" and, therefore, is not limited to high-dimensional models. For example, the sparsity-inducing property of certain high-dimensional estimators (such as the lasso) can be useful for model selection when p is large—irrespective of n.
