Is your input shape [B, L, T*D] corresponding to a channels-last memory format, i.e. would T*D represent the channels? If so, you would have to permute the data, but each kernel would still use all input channels in the default layout.
Could you post the input shape and desired output shapes (with the description what the temporal and channel dimensions would be), please?