Oh yeah, of course. That was silly. Changing the linear layer’s output to [batch size x 256] was all I needed there.
Just a heads up for anyone that comes across this, the code following that linear layer issue here also has bugs, but the @ptrblck’s solution for the linear layer is great.
Thank you!