A possible solution is to split the weight parameter in a layer (e.g. nn.Conv2d) into multiple parameters,
and then concat the outputs together.
For example, for a layer with 2 input channels and 4 output channels, a pseudo code works as bellow:
class MyConv(nn.Module):

    def __init__(self, channel_in, channel_out, kernel_size):
        self.weight1 = Parameter(torch.zeros(1, channel_in, kernel_size, kernel_size))
        self.weight2 = Parameter(torch.zeros(1, channel_in, kernel_size, kernel_size))
        ...
        self.weight4 = Parameter(torch.zeros(1, channel_in, kernel_size, kernel_size))

        ...
    def forward(self, input):
        output1 = torch.nn.functional.conv2d(input, self.weight1, ...)
        ...
        output4 = torch.nn.functional.conv2d(input, self.weight1)
        return torch.cat((output1, ..., output4), dim=1)

And then you can apply channel-wise learning rate tuning by adjusting the learning rates of parameter1, …, parameter4.