Does the word embedding matrix of GPT2 load from the checkpoint,  during the fine-tuning?
Does the word embedding matrix of GPT2 load from the checkpoint,  during the fine-tuning?
（transformer.wte.weight⇒shared token embedding layer Shared weights logic）
  
I tried to do the following:
  
  model = TFGPT2LMHeadModel.from_pretrained("gpt2-medium"）
  model.get_output_embeddings().weight
  
But the following error was occurred, probably because the parameters of wte.weight were not initialized.

  tensorflow.python.framework.errors_impl.FailedPreconditionError: 2 root error(s) found.
  (0) Failed precondition: Error while reading resource variable tfgp_t2lm_head_model/transformer/wte/weight from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/tfgp_t2lm_head_model/transformer/wte/weight)
	 [[node tfgp_t2lm_head_model/transformer/wte/weight/Read/ReadVariableOp (defined at home/nak/cho/.local/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1751) ]]
	 [[tfgp_t2lm_head_model/transformer/wte/weight/Read/ReadVariableOp/_1]]
  (1) Failed precondition: Error while reading resource variable tfgp_t2lm_head_model/transformer/wte/weight from Container: localhost. This could mean that the variable was uninitialized. Not found: Container localhost does not exist. (Could not find resource: localhost/tfgp_t2lm_head_model/transformer/wte/weight)
	 [[node tfgp_t2lm_head_model/transformer/wte/weight/Read/ReadVariableOp (defined at home/nak/cho/.local/lib/python3.7/site-packages/tensorflow_core/python/framework/ops.py:1751) ]]

I am very confused. Shouldn't the parameters of the word embedding table be restored directly from the checkpoint during fine-tuning? Why is it not initialized? 
 
Moreover, when adding special tokens through the following operations, the number of tokens in the dictionary changes, which means the dimension of the word embedding table is changed. Does this mean that the wte.weight needs to be reinitialized? Will the fine-tuning effect be good in this case? 

 num_added_toks = tokenizer.add_special_tokens(special_tokens_dict)
 model.resize_token_embeddings(len(tokenizer))
 
Many thanks!
 
I was mistaken. Sorry.