why the accuracy is very low when we test on sentences one by one of the words input?
# ❓ Questions & Help
hi, friends:
there is a problem , that we trained a good lm model on gpt2 and sequence len is 30,when we batch test in training with 30 seqlen, the accuracy can reach 90%. But when we test like this: 
[1]->[2],[1,2]->[3],[1,2,3]-->[4], left is the input, and we hope it can predict the right, we found the accurcy is very low which is only 6%.


when we were training, the accuracy count method is like this: 
input: [0,1,2,3,4,5,...,29]
label:  [1,2,3,4,5,6,...,30]
so we count the right hits in 30 seqs.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
