You should call the init functions on the specific parameter, i.e. weight or bias.
In this case I would just initialize it after creating the layers or in a separate method:
self.classifier = nn.Sequential(
    nn.Dropout(0.5),
    nn.Linear(16*16*256, 2048),
    nn.ELU(inplace=True),
    nn.BatchNorm2d(2048, eps=0.001),
    nn.Linear(2048, num_classes)
)
        
nn.init.xavier_uniform_(self.classifier[1].weight)
nn.init.xavier_uniform_(self.classifier[4].weight)