Perceptron Criterion


### Inequality
### Linear Classifiers
- Inputs **data **are feature values (**feature vector**) - (x0, x1, x2, ... , xn)
- Each feature has a weight
- Sum is the activation
> ![](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F97f9d462-0b77-464a-8d69-45b5d1a8d92b%2FUntitled.png?table=block&id=18b03cdd-2d54-42ea-8372-c57a7ff78e2a&cache=v2)
- output - if activation is positive → 1 negative → -1 (classification +/-)


### Decision Rule
binary decision rule 
One side corresponds to Y=+1 
Other corresponds to Y=-1

### Learning - binary perceptron
1. Start with weights = 0
> 1. For each training instance
>> - Classify with current weights
>> - If correct (i.e., y=y*), no change
>> - If wrong: adjust the weight vector by adding or subtracting the feature vector. Subtract if y* is -1.

> ![](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fcc561d3b-0dae-4866-bf99-e3d2593dba1f%2FUntitled.png?table=block&id=81c5098b-4384-4058-9615-2678d672d6e7&cache=v2)



## Multiclass dicision rule
![](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2F9af73bdd-20cd-4421-a317-ddd37127ae63%2FUntitled.png?table=block&id=49e83b0c-2442-4df3-acbb-88eefa82cb3f&cache=v2)
lower count of wrong class** (for this instance), **raise count of right class **(for this instance)**
![](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fc30c5970-b6a9-4c98-a062-562a93d94adb%2FUntitled.png?table=block&id=d15b9c2d-8e80-4688-9a70-675be4443d7f&cache=v2)
**black box (kernel) K** that told us the dot product of two examples x and x’
### Solution
0. set true value to +/-
2. get data
3. choose true value
4. classification by hypothesis → compare to true value
5. update hypothesis
to all data try this
until no revise to all data redo these to data set

### multiclass
다른거 똑같은데 분류라서 argmax xf = y
틀리면 그놈은 값 빼주고 맞는놈은 올려주기 f만큼

듀얼이면 argmax alpha K(k is sigma ff) - at sigma only one f change by w
if wrong
틀린놈은 알파에서 1 빼고 맞는놈들은 1만큼 올려줌 맞는 클래스 틀린 클래스
### Feature Vectors
![](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd1bbd38a-1fa3-43b3-96de-b4161118d84f%2FUntitled.png?table=block&id=0d54acc4-2dc6-4fdf-ba33-4f798cdf9949&cache=v2)
Some (Simplified) Biology
- Inputs are feature values
- Each feature has a weight
- Sum is the activation

![](https://www.notion.so/image/https%3A%2F%2Fs3-us-west-2.amazonaws.com%2Fsecure.notion-static.com%2Fd20f5cf0-dbee-4d69-b541-77430ef0677e%2FUntitled.png?table=block&id=ad108811-66aa-45fa-a82f-89af6936407a&cache=v2)
If the activation is: Positive, output +1 Negative, output -1

### Weights
- Binary case: compare features to a weight vector
- Learning: figure out the weight vector from examples



### Decision Rules
Binary Decision Rule in the space of feature vectors
> - Examples are points
> - Any weight vector is a hyperplane
> - One side corresponds to Y=+1
> - Other corresponds to Y=-1






