In GD-optimisation, if the gradient of the error function is w.r.t to the weights, isn't the target value dropped since it's a lone constant? Suppose we have the absolute difference as an error function:
$\mathit{loss}(w) = |m_x(w) - t|$
where $m_x$ is simply some model with input $x$ and weight setting $w$, and $t$ is the target value.
In gradient-descent optimisation, the initial idea is to take the gradient of the loss function, and update $w$ as below:
$w = w - \alpha\cdot\nabla \mathit{loss}(w)$
where the $\alpha$ is the learning rate. Wouldn't the gradient of the loss function in our case be:
$\nabla \mathit{loss}(w) = \nabla m_x(w)$
where $t$ is dropped because it is a constant? I feel I am missing a huge crucial point here.

If we are considering the absolute difference as a norm, that is:
$loss(w) = |m_x(w) - t|$
then $\nabla loss(w)$ is far from simply being equivalent to $\nabla m_x(w)$.
By definition of the derivative for an absolute value (and using the chain rule), we  actually get:
$\nabla loss(w) = \frac{m_x(w) - t}{|m_x(w) - t|}. m_x'(w)$
This is similar Aksakal's answer but I wanted to show exactly why we get $\pm m_x'(w)$
