With regards to the talk of Jakob Uszkoreit, I am wondering whether
symbolic languages like Chinese, which have originated from abstract image
representations of objects might be more likely to generalize in cross-modality joint modeling approaches of image and text, if represented appropriately. In the end also the compositional hierarchy of Chinese ideogram symbols (composed of multiple Chinese characters as radicals to derive a novel character with the ideas from those original characters combined). Since language is likely to have somewhat originated from cave drawings, and those simplified image representations are more closely related to 3D objects in the real world it might be possible to represent languages like Chinese more easily. Maybe by jointly utilizing images of the Chinese characters + token sequence representations of Chinese characters and the real-world image of the object.