Does the biased estimator always have less variance than unbiased one? Suppose I am estimating one of the parameter. Now if we plot the biased estimator of that and unbiased estimator of that can we say for sure that biased one has less variance than unbiased one always.
My thought: my analysis somehow tells me that it is true because if we check bias variance trade off then obviously if we increase the bias from 0 to something, that amount will be deduced from the variance part making the estimator peaky.

NO
Remember that just about anything can be an estimator, even silly estimators. Let’s consider two estimators for $k$ of $\chi^2_k$. Take an $iid$ sample $X_1,\dots,X_n$.
$$
\hat k_1=\bar X \\
\hat k_2=\sum_{i=1}^nX_i=n\bar X
$$
$\hat k_1$ is unbiased, while $\hat k_2$ is biased. However, what are the variances?
$$
\mathbb{V}(\hat k_1)=2k/n\\
<\\
\mathbb{V}(\hat k_2)= 2nk \\
\forall n>1
$$
There’s your counterexample. $\square$
However, I see you making at least two mistakes in your setup.

*

*Multiple estimators can be unbiased. The sample mean, sample median, and first observation (NOT first order statistic) are unbiased estimators for the mean of a normal distribution, for example. (Indeed, the $j^{th}$ observation (NOT order statistic) drawn from a distribution is an unbiased estimator for the mean whenever the distribution has a mean.)


*The MSE does not have to be the same for biased and unbiased estimators. In fact, we tend to pick biased estimators over unbiased estimators because there is such a reduction in variance that the MSE decreases.
EDIT
An even easier example where we’re estimating $\mu$ of $N(\mu, \sigma^2)$:
$$
\hat\mu_1=\bar X\\
\hat\mu_2=\bar X+ 1
$$
Both have the same variance, yet only $\hat \mu_1$ is unbiased.
EDIT 2
If two estimators of a parameter $\theta$, one biased $(\hat\theta_1)$ by some amount $b$ and one unbiased $(\hat\theta_2)$, have the same $MSE$, then it must be that the biased estimator has lower variance. Let $MSE(\hat\theta_1) = MSE(\hat\theta_2) = M$.
$$
(\text{bias}(\hat\theta_1))^2 + \mathbb{Var}(\hat\theta_1) = M = (\text{bias}(\hat\theta_2))^2 + \mathbb{Var}(\hat\theta_2) $$$$
b^2 + \mathbb{Var}(\hat\theta_1) = 0 + \mathbb{Var}(\hat\theta_2) $$$$
\mathbb{Var}(\hat\theta_1) = \mathbb{Var}(\hat\theta_2) - b^2$$$$
\implies\mathbb{Var}(\hat\theta_1) > \mathbb{Var}(\hat\theta_2)
$$
(I confess that I don't know what happens if we work over complex numbers (so that $b^2<0$ is possible), though I would imagine that the decompision of $MSE$ in that case is something like $MSE(\hat\theta) = (\text{bias}(\hat\theta)) \overline{(\text{bias}(\hat\theta))} + \mathbb{Var}(\hat\theta)$.)
