How does gradient descent help SVM learn a linearly separable hyperplane? So I see the Perceptron Algorithm applied to learning an SVM, where $\theta$ is the normal vector to the linearly separating hyperplane. How does the update
$$\theta^{t+1}\leftarrow\theta^t+\alpha y_ix_i$$

There's this visual example, but I'm not sure how updating $\theta$, causes it to adjust in the correct direction.
Help push the hyperplane to learn a linear separation between the points $x_n$, where $y_i$ and $x_i$ are misclassified points?

According to the update equation (below the figure), there is vector subtraction. To perform it, just draw a line from the point to the tip of $\theta$. Your new $\theta$ would be this vector, so move it to the origin as others, and draw a normal as your hyperplane. This would be the case for $\alpha=1$, i.e. $\theta-x$. Assuming a smaller $\alpha$, e.g. $0.5$, this subtraction would produce a vector more like the one on the right.
