Solved by SrJ in post #6 
              
              
                Ok I have found the solution to my problem. It is with the Optimizer. As i have used a distilBert Layer at the beginning , i have to use very low lr  like 3e-5 according to the paper.