Understanding Weight Updates in a Perceptron I am learning about perceptrons and how they work. I read that each weight $w_j$ is updated based on the equation:
$\begin{equation}
w_j:=w_j+\Delta w_j 
\end{equation} $
Where:
$\begin{equation}
\Delta w_j= n \times (y^{(i)}-\hat{y^{(i)}}) \times x_j^{(i)}
\end{equation}$
Here, $n$ is the learning rate. There are two things that are unclear to me. Namely, why do we take $x_j^{(i)}$ (the input) into consideration? Would not this cause problems when different features have different scales? I know standardization is a potential solution but this was not mentioned. Moreover, it would also lead to vastly different weights when a single feature can take a large range of values (example: a pixel with red value 0 and a pixel with red value 255 on the RGB scale). Moreover how are class labels $\hat{y^{(i)}}$ represented? Would not choosing numbers 0,1,2... create the same type of problem mentioned above ?

First of all, using the input variable is consistent with the derivative based gradient descent approaches. Other than this, intuitively, perceptron algorithm adds the input variable to the weights such that it tries to make the weights closer/farther to itself. Closer when its label is $1$, because the update is $r(1-\hat y)x$ and farther when its label is $0$, because the update is $r(0-\hat y)x=-r\hat yx$. When weight and input vector become closer, their dot product will be larger, and the predicted label will be $1$. When they become farther (e.g. in negative directions, $w=-x\rightarrow$ $wx=-||x||^2$), their dot product will be smaller and the predicted label will be $0$. This is why input feature plays an important role on the updates.
This principle still works when input features are of different scales. However, the learning may be slow, or the final result might be different. So, normalization is usually recommended.
In multi-class problems, we have different neurons with binary outputs. The class labels are converted into one-hot vectors and compared with the outputs from these output neurons. The practice of class labels being ordered integers is rare and it also implies an implicit ordering between the classes. Similarly, it can cause problems with the activation function choice.
