Hey @nielsr is DocFormer 5 currently on your roadmap for transformers?
@iakarshu is thinking about having a go at implementing and pretraining it (because the authors didn’t release code or weights), so I thought it would be good to double-check that you don’t do the same work twice