I think,  you can use conv2d to achieve what you want. Something like:
data = torch.randn(150, 96, 96, 240).cuda()
conv = nn.Conv2d(240, 1)
output = conv(data.permute((0,3,1,2))).squeeze()

P. S.  I didn’t check for syntax errors.