Saving full tensor output of hidden states instead of truncated output in lm_finetuning.py script
Hi,

The past few weeks I have been playing around with the "run_lm_finetuning.py" script to finetune a custom dataset and extract it's features by setting 'output_hidden_states=True' and saving the features of the second last layer by changing the code of the script as follows:

`model.train()`
`outputs = model(inputs, masked_lm_labels=labels) if args.mlm else model(inputs, labels=labels)`
`loss = outputs[0]  # model outputs are always tuple in transformers (see doc)`
`torch.save(outputs[-1], 'output.pt')`

The tensor data gets truncated as follows and I have not been able to figure out yet how to save the full tensor output:

> [[ 0.0656, -0.1678, -0.4601,  ...,  0.0111,  0.0955,  0.7106],
         [ 0.7000, -0.5496,  0.6127,  ...,  0.0038,  0.3024, -0.2240],
         [ 0.1105,  0.3366,  0.1706,  ..., -0.1861, -0.0499,  0.0265],
         ...,
         [-0.3434, -0.1283, -0.0637,  ..., -0.2911, -0.7759,  0.0511],
         [ 0.3330,  0.3573, -0.2226,  ...,  0.4622, -0.6238, -0.5374],
         [ 1.1726,  0.0471, -0.0415,  ...,  1.3879, -0.3199,  0.2052]]]

I have been trying to figure it out by myself because I know the problem is my lack of experience with Python and Pytorch but really been hitting a wall trying to figure this one out.

Can anyone point me to the right direction how to save the full tensor output?
Hi, how are they truncated? When you call `tensor.shape`, is it not the shape you're expecting?<|||||>@LysandreJik thanks for mentioning tensor.shape. I was so convinced the data was truncated and forgot to check the shape to confirm. Because of your hint I realized the output that gets printed is truncated but the actual data itself is not and is fully there.

Thanks ;)