I’m also interested in this task 
I would like to join forces if you are interested. I’m pretty confident in NLP.
I know CLIP, (some of) vqgan model but don’t know Jax.
My github: cceyda 25
Also see this dataset 11