@sshleifer @patrickvonplaten @valhalla
I’m now trying to implement the method, and it seems my implementation works on CPU.
(I had to set --max_source_length to a small value for avoiding index out of range error.)
I saw that the code occurred errors (CUDA error, not reproducible…) on GPU, but it sometimes worked without error. I think I should check the reason.
I’m sorry but I haven’t tried it on TPU.
I’m now working on branch forest1988-prophetnet-prepare-seq2seq-batch of https://github.com/forest1988/transformers.git .
I’ll continue to check if it works as I intended on GPU and TPU.
Moreover, I made a ipynb file for debugging and it maybe useful for you to check how my modified implementation works on Colaboratory.
https://github.com/forest1988/colaboratory/blob/main/prophetnet_seq2seqtrainer.ipynb 4 .
Thank you.
yusukemori