This code snippet does the expected splitting:
class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        pass

    def forward(self, feature_batch, tensorA):
        print(feature_batch.shape, feature_batch.device)
        print(tensorA.shape, tensorA.device)
        return feature_batch

model = MyModel().cuda()
model = nn.DataParallel(model, device_ids=[0, 1])
feature_batch = torch.randn(32, 128).cuda()
tensorA = torch.randn(500, 128).cuda()
out = model(feature_batch, tensorA)
> torch.Size([16, 128]) cuda:0
torch.Size([250, 128]) cuda:0
torch.Size([16, 128]) cuda:1
torch.Size([250, 128]) cuda:1

Are you sure you haven’t increased the batch size?
Anyway, if you want to pass the same tensorA to all replica, you could use:
tensorA = tensorA.unsqueeze(0).expand(2, -1, -1)
out = model(feature_batch, tensorA)