I solved it. Error was in my code which is related to number of classes. Actually I have given the vocab size manually and did not consider index 0 as of one of the class.
Thanks @ptrblck for your instant replies.