First thing to do is make a clear distinction between loss and error. The loss function is the function an algorithm minimizes to find an optimal set of parameters during training. The error function is used to assess the performance this model after it has been trained. We always minimize loss when training a model, but this won't neccessarily result in a lower error on the train or test set. In fact, you could argue that we are deliberatly increasing error on the train set when we are trying to prevent overfitting. Obviously we would ideally like to minimize error on the test set, but this isn't the same as minimizing loss. Minimizing the error involves selecting good features, the appropriate model, fine tuning hyperparameters, etc. whereas loss is minimized during training, regardless of what features or model you use.
Now I'll try to address you're question on partial derivatives in 2 parts. One part telling what are the criteria to find a minimum and another part to explain why these are the criteria to find a minimum
What are the criteria to find the minimum of a loss function
Suppose you have a loss function $L(\omega)$ which is a function of the parameters
$$\omega= \begin{pmatrix} \omega_{1}\ \omega_{2}\ \vdots \ \omega_{N} \end{pmatrix} $$
The local minima of such a function are found by finding values of $\omega_j$ which satify the following
\begin{equation} \nabla L = 0 \end{equation}
and \begin{equation} det(H_L) \ge 0 \end{equation} where $\nabla L$ denotes the gradient of $L$ and $det(H_L)$ denotes the determinant of the Hessian matrix of second partial derivatives of $L$ This wiki page has further details https://en.wikipedia.org/wiki/Hessian_matrix#Second-derivative_test
However, it's generally assumed that the function $L(\omega)$ is a convex function(most loss functions you'll see are in fact convex). One of the properties of convex functions is that $det(H_L)\ge0$. 915
So for convex functions we only need to solve for $\nabla L = 0$
Wiki page on convex funtions: https://en.wikipedia.org/wiki/Convex_function#Functions_of_several_variables
Why are these the criteria to find the minimum of a loss function?
But another question you might have from reading this is:
"You've told me how to find a minimum, but you haven't told me why these criteria find a minimum"
To think about this remember what $\nabla L$ tell us about the function $L$. $\nabla L$ tells us about the "steepness" of $L$, i.e. the vector $\nabla L$ points in the direction where $L$ is steepest and the norm of $\nabla L$ tells you how steep $L$ is. So when $\nabla L=0$ for some value of $\omega$ it means, $L$ is "flat" in every direction for that value of $\omega$.
As for $det(H_L)$, this value (kind of) tells you how $\nabla L$ itself changes. So when $det(H_L)\ge0$ it mean $\nabla L$ will increase in every direction. But based on what we said previously, if $\nabla L$ is increasing in every direction, then the "steepness" of $L$ will increase in every direction.
So let's put these 2 observations together. If $\nabla L = 0$ and $det(H_L)\ge0$ for some value of $\omega$ it means for that particular value of $\omega$, $L$ is flat but the steepness of $L$ will go up if you move in any direction. So $L$ must form a sort of bowl shape where $\omega$ identifies the bottom of the bowl. But the bottom of the bowl is the minimum value of $L$. Therfore $\omega$ is where $L$ is minimized.
Hope this helps 418