So basically what happens is the first number that is passed to the linear layers is the batch size. The way the linear layer works is it computes the outputs over all batches so the batch number doesn’t change only the input number. If you want something of shape [28, s] you would need to put the 28 first as the batch size.