Sorry, but the requirement is that the input needs to be fed in as 30 samples of [16, 2] matrices, and the context vector after encoding needs to be [256, 1]. Cann’t change that.
If it’s not possible in torch, gotta do it in tensorflow.