Does the biased estimator always have less variance than unbiased one? Suppose I am estimating one of the parameter. Now if we plot the biased estimator of that and unbiased estimator of that can we say for sure that biased one has less variance than unbiased one always.
My thought: my analysis somehow tells me that it is true because if we check bias variance trade off then obviously if we increase the bias from 0 to something, that amount will be deduced from the variance part making the estimator peaky.

I don't think there's an answer as to which will have more variance.  Take the case of the sample variance. The unbiased estimate
is $\sum_i (x_{i} - \bar{x})^2/(n-1)$. It has variance
$\left(\frac{1}{n-1}\right)^2$ Var$\left(\sum_i (x_{i} - \bar{x})\right)^2$
The biased estimate has variance:
$\left(\frac{1}{n}\right)^2$ Var$\left(\sum_i (x_{i} - \bar{x})\right)^2$
So, in this case, the biased estimate will have less variance. Each case probably depends on the estimate and a generalization as to which has or more less variance is probably not possible.
