Question: I have four samples

```
x1 = [1,1,2,5]
x2 = [1,1,1,2]
x3 = [1,5,5,5]
x4 = [1,1,2,2]

```

I am looking for a way to test whether the four populations are siginificantly different from each other. The only possible symbols the sample are 1, 2, 5. The symbols are ordinal.

Is there a hypothesis test that does this? I was thinking of pair-wise application of students t-test, but i cannot make any normality assumptions. Ideally, I am looking for an implementation in python.

Answer: 
I find it helpful to think about logistic regression as a special case of Generalized Linear Models (GLM). In GLMs, we assume that the conditional distribution of the response belongs to the exponential family of distributions. (I say the "conditional" distribution here because the response follows this distribution only for some fixed value of the independent variables and the parameters). We then further assume that the expected value of the conditional distribution of the response (or the conditional expected value) is given as a function of a linear combination of the independent variables, $h(x^T\theta)$. The inverse of this function, $h^{-1}$, is called the link function.

In logistic regression, we assume that the response $Y$ is conditionally distributed following a Bernoulli distribution (a binomial distribution with $n=1$), where the parameter $p$ (the "success rate") may depend on $x$ and $\theta$, thus $$Y|x;\theta \sim Bern(p(x,\theta)).$$
We now wish to choose $p(x,\theta)$ in such a way that the conditional expected value of $y$ is equal to $h(x^T\theta)$, $$E(Y|x,\theta) = h(x^T\theta).$$ Conveniently, in the Bernoulli distribution we have that the expected value is just equal to $p$, which is equal to the probability that $Y = 1$. Therefore, $$E(Y|x,\theta) = P(Y=1|x;\theta)= h(x^T\theta).$$
If we define $h$ to be the standard logistic function, $$h(x^T\theta)=\frac{1}{1+e^{-x^T\theta}},$$
your case follows.
We are not simply using the hypothesis from linear regression.
And we plug it into the sigmoid function to convert into a new hypothesis for a couple of reasons!

Before we start we must know that the Logistic Regression Model is applied to Classification Problems.
Classification can be of two types: (1) Binary- (1 or 0), (True or False), (Yes or No) etc.
(2) Multiclass- (1, 2, 3 or 4) etc.

Ans:-
Now we substitute this hypothesis into the sigmoid function because:
Logistic Regression Model wants 0 <= hθ(x) <=1
A Sigmoid function helps us convert hθ(x) to values between 0 and 1

[![Sigmoid Funtion](https://i.stack.imgur.com/nkGi9.png)](https://i.stack.imgur.com/nkGi9.png)

This is where Probability comes into the picture.
The Hypothesis hθ(x) = P(y=1|x;θ)

Here hθ(x) = estimated probability that y=1 on input x.
That is the sigmoid function directly provides us with this probability, as it has a range of [0,1] just like probability has [0,1] in mathematics.

[![Sigmoid Function applied for Logistic Regression with two values of y](https://i.stack.imgur.com/w1yNS.png)](https://i.stack.imgur.com/w1yNS.png)

The reason why we don't use Linear Regression for Classification:

For example- We want to predict a tumour is Malignant or Benign
using one feature-x: Tumor size
and we want to predict y: Malignant(1) or Benign(0).
We can threshold the classifier output of hθ(x) at 0.5 as follows:
If hθ(x) >= 0.5, predict 'y=1'
If hθ(x) < 0.5 'predict y=0'

But if we apply this same technique with more training data:

1. it may predict wrong.
2. it may predict y>1 or y<0, but we need either 0 or 1 as answer in y.
So, applying Linear Regression to a classification problem often isn't a great idea.