I don’t think there is any specific formula for fixing on embedding size. Like you said, it could be decided based on experimentations. Incase of images, the embedding size can vary based on the input image size, architecture being used.