We Could train NLP based models to generate embedding for sentences and then use this embeddings as input embedding for gan model to generate images. This is pretty simple idea just vice versa of image captioning, but I think it can work pretty well given the performance of transformers based models.