Question: I'm studying statistics, and in a lot of textbooks, the regression formulas always refers to the functions themselves as f(x) parameterized by a,b,c or something. And they are often written $f(x; a,b,c)$, why can't we just write $f(x, a, b, c)$?

I would like to know perhaps the history of how this came about, and why there is this seemingly dual method of expressing the same idea. Is there some esoteric mathematical notation that a noob like me hasn't encountered yet, or did mathematicians of old just like doing things in different ways?

An example: A function that maps age of tree to height:

$$f(age; growth\\_rate) = age \* growth\\_rate$$ vs $$f(age, growth\\_rate) = age \* growth\\_rate$$

Answer: 
Strictly speaking you're right. It's sometimes difficult to explain clearly what is a *parameter* and what is a *variable*. When you write $f(x;a,b,c)$, there is no mathematical difference between $x$ and $a$ for example, they're both *variables* for the function $f$. I think the real difference is in how we interpret things. When we have an expression such as $f(x;\theta)$ or $f\_\theta(x)$, $x$ and $\theta$ don't exactly play the same role, that's why we use different notations for them. In fact it's for the sake of clarity. When you have lots of different variables in an expression, using different notations is an easy way to know what role they play.

In the framework of statistics, the notion of *parameter* is even more important. As you may know, there's a whole branch of statistics called *parametric statistics*, where we introduce families of probability distributions, indexed by a set of *parameters*:
$$\mathcal{P}=\{P\_\theta,\theta\in\Theta\}$$
where $\Theta$ is some subset of $\mathbb{R}^d$.
Note that you could actually see this as a function from $\Theta$ to the set of probability distributions on a certain space. In many cases, $P\_\theta$ has a density w.r.t. the Lebesgue measure, and we denote it by $f(x;\theta)$ or $f\_\theta(x)$. You see in this example that $x$ and $\theta$ have very different meanings. $\theta$ is the parameter that comes from our statistical model, whereas $x$ is a variable of integration.

In other cases, we use the "opposite" notation. For example when you consider the likelihood function $l(\theta;x\_1,\dots,x\_n)$, we consider it primarily as a function of $\theta$ (that we want to maximize most of the time), and $x\_1,\dots,x\_n$ as *fixed* parameters (they are the observation, we can't change them). Once again, it's a question of context and meaning of the variables.
Think to something like this: $f(x;\sigma) = e^{-x^2/\sigma^2}$.

If you read for example "the derivative of $f$" probably you immediately undestand that we are speaking of $\partial f/\partial x$ because you recognize $f$ as a function of $x$, whose definition contains some $\sigma$ which is not a "variable".

Otherwise if you read:
$f(x, \sigma) = e^{-x^2/\sigma^2}$
a sentence like "the derivative of $f$" would be ambiguous: it's $\partial f/\partial x$? $\partial f/\partial \sigma$? $\nabla f = (\partial f/\partial x, \partial f/\partial \sigma)$?

So without this distinction every time one has to write explicitly "the derivative of $f$ with respect to the variable $x$".