This may be not entirely about NLP.
I am working on Image captioning and learning textual representations from CNN features.
Idea is to train CNN using captioning. So, I tried to use GPT-2 tokenizer but I had to create Captioning model from scratch.
Is there any way to attach TF Transformer models to other CV applications for better learning?
My VirTex implementation in Keras 7