ptrblck:

unfold


I dont’t know how to profile the backward steps since the time bottleneck happens in the backword. To solve the problem, I use the functions implemented in the pytorch. By replacing the windows abstraction and matrix multiplication with nn.Conv1d, the speed in backward is far faster than before.
class CircleConv(nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size=3, stride=1):
        super(CircleConv, self).__init__()
        assert kernel_size // 2 != 0, 'kernel size should be an odd!'
        self.kernel_size = kernel_size
        self.kernel_size_number = kernel_size * 1
        self.out_channels = out_channels
        self.conv1 = nn.Conv1d(in_channels=in_channels, out_channels=out_channels, kernel_size=3, stride=stride, padding=0)

    def forward(self, x):
        length = x.shape[-1]
        half = self.kernel_size // 2
        assert length>=half, 'length is shorter than half kernel!'
        triple = torch.cat(tensors=[x[:, :, -half:], x, x[:, :, :half]], dim=-1)
        result = self.conv1(triple)
        return result


The time record is as follows:
forward eclapsed time:1.2164795398712158
backward eclapsed time:0.11817550659179688
forward eclapsed time:0.009972095489501953
backward eclapsed time:0.025435686111450195
forward eclapsed time:0.009973287582397461
backward eclapsed time:0.02593064308166504
forward eclapsed time:0.009973287582397461
backward eclapsed time:0.02593064308166504
forward eclapsed time:0.009973287582397461
backward eclapsed time:0.02593088150024414
forward eclapsed time:0.009973526000976562
backward eclapsed time:0.024933338165283203
forward eclapsed time:0.009972810745239258
backward eclapsed time:0.0249330997467041

With the limited knowledge on the programming, I don’t understand what results in the time consumption in the backward progress.