Thank you very much! That was very useful. 
Although, only the bias_filler has been set to type:‘constant’ and value: 0 , implying they initialize it with 0. They have bias for the last layer as well.
But I regret I did not see the ‘prototxt’ files as closely as I should have. I found two things that I wasn’t doing:

The learning rate for Bias is twice the learning rate for the corresponding weights in all layers.
The weight decay for  all Bias parameters is kept 0.

I wish such details are not missed out in the papers. No reason for this to be lesser important than some of the other implementation details that they share.
Again, Thanks a lot for pointing this out!