Question: Can anyone explain if it's good to have the Gnd pin of a Bypass Caps tied to IC's Gnd? I've observed some design using those methods. Just want to have a justified answer. 

I'm not sure if the thickness of the board plays a huge role in here.

Answer: 
> 
> if it's good to have the Gnd pin of a Bypass Caps tied to IC's Gnd? 
> 
> 
> 

Absolutely, if the capacitor ground isn't connected to the IC ground, it can't do its job of providing a low-impedance path between those two nets.

If you want to think of it in terms of ground loops, think about it this way:

If the capacitor wasn't there, then when the IC draws current it would have to draw it from the power supply itself, or from a bulk capacitor. In either case, the part supplying the current would be much further away. 

That means the current would flow in a loop from the power supply to the IC and back to the power supply. That would be a much larger loop than from the very close-by bypass capacitor to the IC and back.

So what the bypass capacitor is doing is shorting out a large loop by making a small one. Which reduces the emissions from the loop and reduces the inductance of the loop, causing less voltage ripple as the current demand changes.
It's important to have any required high frequency decoupling capacitor as close to the IC pins as possible. That capacitor supplies short bursts of current to the chip (generalism alert) that would otherwise be provided from a capacitor a bit further away and fed down copper tracks. Keeping tracks short means keeping EMI and circuit cross talk down.

Some chips will just not perform if their respective decoupler caps are not close up at the chip. We're talking 100 nF and under here to fall into this type of category.