What does "tail" mean in probability distribution? And what do we mean by heavy or light tail? I have seen this word so many times while studying the probability distributions, till I should probably want to know exactly what it means, to boost my understanding of probability concepts.

You can find the relevant definitions
here
In essence, a distribution is called heavy-tailed if the tail probability $P[X > x]$ decreases slower then exponentially. Formally :
$\lim_{x\to\infty} e^{tx} P[X>x] = \infty $ for all $t>0$
