Ahh yes, I wasn’t thinking of the subsequent softmax operation. We want the elements of softmax output corresponding to future tokens to be 0, and not the dot product itself. Thanks for the prompt response!