Does the biased estimator always have less variance than unbiased one? Suppose I am estimating one of the parameter. Now if we plot the biased estimator of that and unbiased estimator of that can we say for sure that biased one has less variance than unbiased one always.
My thought: my analysis somehow tells me that it is true because if we check bias variance trade off then obviously if we increase the bias from 0 to something, that amount will be deduced from the variance part making the estimator peaky.

There are many, many, many different possible estimators in estimation problems.  In general there are multiple unbiased estimators and multiple biased estimators, and their variances need to be considered on a case-by-case basis.  A biased estimator may have a lower, or higher, or the same variance as an unbiased estimator.  Now, if we consider estimators on the basis of their mean-squared-error (MSE), then we will generally consider the lower bias and higher variance as a trade-off and we will ignore any estimators that have both higher bias and higher variance than another estimator.  In this context we are left with estimators where the unbiased ones have higher variance and the biased ones have lower variance.  Nevertheless, this is a contextual result that comes from setting other estimators out of consideration.

As a counter-example to show that it is possible to have an unbiased estimator with higher variance than a biased estimator, consider a set of data values $X_1,...,X_n \sim \text{N}(\mu, 1)$ and consider the estimators:
$$\hat{\mu}_1 \equiv X_1 + b
\quad \quad \quad \quad \quad
\hat{\mu}_2 \equiv \bar{X}_n,$$
where $b \neq 0$.  It is simple to establish that $\hat{\mu}_1$ is biased and $\hat{\mu}_2$ is unbiased, and their respective variances are:
$$\mathbb{V}(\hat{\mu}_1) = 1
\quad \quad \quad \quad \quad
\mathbb{V}(\hat{\mu}_2) = \frac{1}{n}.$$
If $n>1$ (i.e., if you have more than one data point) then the first estimator has a higher variance.  Since this is the biased estimator, it is both biased and has higher variance than the unbiased estimator.
