I think you have to implement a new nn.Module subclass for the ConvLSTM and add the parameters corresponding to the W and b parameters to the module parameter pool. Then it is just a matter of implementing the math with the available pytorch operations…
In order to make it easier, maybe it would be better to implement a ConvLSTMCell first that just produces one time step output and then the full class ConvLSTM that deals with the loop and does the call to ConvLSTMCell.