What you mean by fuse is very confusing and unclear.
For example:



 Ganga:


So I want to try to fuse(learnable parameter) ( Linear->Relu )



Is very confusing, since the normal ReLU is a classical activation function defined as ReLU(x)=max(0,x)  and has no learnable parameters (unlike activation functions like PReLU).
I think everything that can be said was already said by @JuanFMontesinos
If this is not the answer you are looking for



 JuanFMontesinos:

Something like the following?
import torch
from torch import nn


class BNLinear:
    def __init__(self, in_features: int, out_features: int, bias: bool = True):
        self.linear = nn.Linear(in_features, out_features, bias)
        self.batch_norm = nn.BatchNorm1d(in_features, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

    def forward(self, x):
        x = self.batch_norm(x)
        return self.linear(x)



then this is probably your answer:



 JuanFMontesinos:

Well not in principle. The weights itself are dedicated to learn different stuff. There are not even the same amount of parameters.