Novice here. I am trying to implement convnet on top of huggingface bert-base-uncased, and I am using a tensorflow example. The original model looks something like:
tf_roberta_model (TFRobertaMode ((None, 96, 768), (N 124645632   input_1[0][0]

dropout_38 (Dropout)            (None, 96, 768)      0           tf_roberta_model[0][0]


conv1d (Conv1D)                 (None, 96, 128)      196736      dropout_38[0][0]

I have a couple questions:

how can an input shape of (None, 96, 768) after being applied: tf.keras.layers.Conv1D(128, 2,padding=‘same’)(x1) output a shape of  (None, 96, 128) ?
I’m struggling to implement the same as above in pytorch, and so far I got it to run (albeit does no good) as follows: nn.Conv1d(in_channels=128, out_channels=1,kernel_size=2). What am I doing wrong?

Thanks in advance!!!