Thanks for the code snippet.
The error is raised, as torch.cat(batch_state) and torch.cat([s for s in batch.next_state if s is not None]) will return the flattened tensors. Use torch.stack instead and the code should be working fine.
I haven’t checked the exact differences between your code and the tutorial, but guess your states are containing multiple state values, which would yield this error.