What makes training time longer with bigger parameter size in a deep learning model? I try to understand, is it always the case with the more parameter you trained, the more training time you need when training a deep learning model. For example, i have a CNN model for text classification. I'm comparing two embedding (word2vec and fasttext) with different embedding dimensions, 512 for word2vec and 300 for fasttext. The result is model with word2vec has more parameter size, and the training time is also longer. Is this always the case? Simply because of the number of the parameter? Because the CNN architecture is the same, the only difference is the word embedding used.

It depends on the initialization point, your configuration etc as well, but no surprise it's usually the case. Number of parameters increases the dimensionality of the parameter space you search for, and it's less likely to fall in to the right place if you are in a larger space, e.g. think about a point in 2D vs 3D spaces, which one is easier to get near when you're randomly assigned a coordinate? Moreover, more parameters/inputs means more calculations, so, no wonder if it increases the training time.
