#model1
def getmodel():
    model = EfficientNet.from_pretrained('efficientnet-b5')
    model._fc = nn.Sequential(
         nn.Linear(in_features=2048, out_features=1024, bias=True), #first layer 
         nn.ReLU(),
         nn.BatchNorm1d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True),
         nn.Dropout(p=0.5),
         nn.Linear(in_features=1024, out_features=4, bias=True)) #last layer
    model = model.to(device)
    return model

I have lot of question from the above code block which I want to clarify.

I have read that bias should be True (bias=True) at the last linear layer. And my model also performed well when turned on.
Most people suggested that bias should be turned off (bias=False) before using batch norm ( Even bias in the Conv layers of EfficientNet are turned off before batch norm). But my model performed badly when I turned off the bias at the first layer. What should I follow?
This model (model2) just uses the single linear layer.

#model2
def getmodel():
   model = EfficientNet.from_pretrained('efficientnet-b5')
   model._fc = nn.Sequential(
        nn.Linear(in_features=2048, out_features=4, bias=True)) #last layer
   model = model.to(device)
   return model

Whcih approch should I follow model1 or model2 ?