Cant see 184 in the input size, from where this came from.
The issue is with the size, please check forward function also .
Something is happening here
x = x if sample else torch.cat((x, self.init_padding), 1)
print the sizes in forward function it will help to debug.