How difficult would it be to integrate a text to image generator like Stable Diffusion with a large language model like Open Assistant?
It depends on the specific implementation of the Large Language Model. Some LMs are designed to work well with NLP tools, while others may require significant modification or additional training data in order for them to produce high-quality results.