This is also the solution I thought of but the problem is a hidden vector is float32 in general and can contain negative values. Multiplying by -inf gets +inf. The padding tokens can be assumed to be 0 for simplicity.