Really，really thanks for your help!
Actually I am a student from China and I get these codes at a chinese cooding net. You don’t need to update it 
But I still have the question, actually I want to get the word that my last_hidden_state refer to. There are 7 words in input sentences. And I actually get the mean vector of them, so the size is [1,768]. I want to “decode” it to the word that it refers in dictionary. 
Usually in bert, we first change words to one-hot code by dictionary provided and then we embed it and put the embedding sequence into encoder. I want to “de-embed” the tensor out of the bert, which is use this tensor class the transpose of embedding matrix. But how can I get the transpose of the matrix.
The second question is that, actually the document did not provide enough guide code to let us know the strcture of model(may be I am too weak).