Shouldn't GPT2 use Linear instead of Conv1D?
Conv1D seems to be inherited from GPT but does not seem to serve any special purpose in GPT2 (BERT uses Linear).

Should GPT2's model be moved to using Linear (which is easier to grasp obvioulsy)?
Maybe this would break pre-trained weights loading? Interested to understand if that's the only reason?<|||||>Possibility, feel free to test the modification and submit a PR @spolu!<|||||>Hi guys,
I also wondered whether anyone modified the gpt2 model to have nn.Linear instead of Conv1D layers (using the pre-trained weights).
Did any of you success or found such implementation?