Solved by ptrblck in post #2 
              
              
                This should yield the same result: 
q = np.random.randn(150, 96, 120)
k = np.random.randn(150, 96, 120)
batch_size, sent_len, vec_size = q.shape  # i.e. [150, 96, 120], also k is same
q_ = np.repeat(q, sent_len, axis=1)
k_ = np.tile(k, (sent_len, 1))
concat_vecs = np.stack([q_, k_], axis=2).reshape(…