Does the biased estimator always have less variance than unbiased one? Suppose I am estimating one of the parameter. Now if we plot the biased estimator of that and unbiased estimator of that can we say for sure that biased one has less variance than unbiased one always.
My thought: my analysis somehow tells me that it is true because if we check bias variance trade off then obviously if we increase the bias from 0 to something, that amount will be deduced from the variance part making the estimator peaky.

It is certainly not the case that all biased estimators have less variance than unbiased ones. That is quite trivial to see: pick any unbiased estimator, then create a new estimator that consists of that estimator plus some random non-zero mean random variable. It is also not the case the lowest variance biased estimator always has lower variance than the lowest variance unbiased estimator. For instance, given standard linear regression assumptions, OLS gives a zero bias estimator that has the lowest overall variance.
However, if the lowest variance unbiased estimator has a lower variance than the lowest variance biased estimator, then there probably isn't any reason to use the biased estimator, so you aren't going to hear about the biased estimator. That is, if estimator with the lowest variance overall is unbiased, then people are going to just use that one, and there's no reason to examine which biased estimators have the lowest variance. So ... I guess you can say that you have a biased view of biased estimators, because you only hear about the ones that have lower variance than the unbiased estimator.
