Got an idea for this question. I can possibly apply only convolution layers to the entire signal and feed the feature map into LSTM model. This should form much clearer code. However, it may violate the requirement batch processing. Any insights about this problem?