Hi !
You have done a great work, I am also interested in CLSTM and want to do something using it.
I don’t know how it run in your machine, but I can’t run your code directly, so I rewrite some parts and it can run well with these changes, I changed the loop in CLSTM.forward to:
for idlayer in xrange(self.num_layers):
    hidden_c=hidden_state[idlayer]
output_inner=[]

for t in xrange(seq_len):
    hidden_c=self.cell_list[idlayer](current_input[:,t,:,:,:],hidden_c)
    output_inner.append(hidden_c[0].unsqueeze(1))

next_hidden.append(hidden_c)
current_input=torch.cat(output_inner,1)

Does these changes conflict with your original intension?