Ok, with variable input lengths, this makes some sense. But you’ll probably need a deep network (MLP) for 1d->5d transformation, as 1x5 linear map just represents a line in 5d space, not sure if that’s of any use with your aggregation method… In most scenarios, another linear map 5xH will be applied to that 5 column tensor, so at least some non-linearity is needed
(otherwise (input @ 1x5) @ 5xH = input @ (1x5 @ 5xH) = input @ (1xH), so first linear layer is not needed).