I have tried also by converting the 1d signal to 2d and then using nn.Conv2d to share the kernels across streams of time series. It worked as I wanted but that somehow consumes huge amount of memory (despite the reduced number of parameters in the model).