Programming, Artificial Intelligence, Machine Learning, Data Science, Python.
directly affects the size of the word embedding matrix and the input/output dimensions of the linear layers in the feed-forward network. Increasing the num_heads (number of attention heads) increases the number of parameters in the multi-head attention layer. Modifying the feedforward_dim