I want to get per sentence vector from word embeddings by max pooling, with seq length(mask).
The code like below, similar strategy works well using tf. and using pytorch using masked softmax also ok.
But below code will cause nan, why ?
class MaxPooling(nn.Module):
def forward(self, x, x_mask):
if x_mask is None or x_mask.data.sum() == 0:
return torch.max(x, 1)[0]
else:
x_mask = x_mask.unsqueeze(-1).expand(x.size())
x.data.masked_fill_(x_mask.data, -float(‘inf’))
return torch.max(x, 1)[0]