I like the idea of stride over the depth dimension, never thought about something like that.
But I still don’t get how this helps to reduce dimensionality.
I build a small example:
import torch.nn as nn 
import torch

a = torch.rand((6, 3, 2, 217, 512)) # bs, timesteps, channels, H, W
a = a.transpose(1,2) # swap timesteps and channels
conv = nn.Conv3d(in_channels= 2, out_channels= 2, kernel_size=1, padding = 0)#<-- Out channels = 2
b= conv(a[:,:,0:1,:,:])
c= conv(a[:,:,1:2,:,:])
d= conv(a[:,:,2:3,:,:])
for i in [a,b,c,d]:
  print(i.shape)
#torch.Size([6, 2, 3, 217, 512])
#torch.Size([6, 2, 1, 217, 512])
#torch.Size([6, 2, 1, 217, 512])
#torch.Size([6, 2, 1, 217, 512])


I end up with b, c, d which are again 3 tensors I would need to stack which ends up at the original.
What I could do is something like:
a = torch.rand((6, 3, 2, 217, 512))
a = a.transpose(1,2)
conv = nn.Conv3d(in_channels= 2, out_channels= 1, kernel_size=1, padding = 0) #<-- Out channels = 1
b= conv(a[:,:,0:1,:,:])
c= conv(a[:,:,1:2,:,:])
d= conv(a[:,:,2:3,:,:])
e = torch.cat((b,c,d), dim=2).transpose(2,1)
for i in [a,b,c,d,e]:
  print(i.shape)
#torch.Size([6, 2, 3, 217, 512])
#torch.Size([6, 1, 1, 217, 512])
#torch.Size([6, 1, 1, 217, 512])
#torch.Size([6, 1, 1, 217, 512])
#torch.Size([6, 3, 1, 217, 512]) <--goal dimension

Do you have some paper/report/blogpost that investigated this striding? Or was this suggestion based on your personal experience?