Hi @dav-ell.
For the training, you pass the complete inp_emb and tgt_emb updated with positional embeddings. tgt_emb is your ground-truth embeddings. The mask is computed by the Transformer module to have access to only the previous outputs in time.
However, during inference you don’t have access to the ground truth tgt_emb and have to input output[i-1] as the tgt_emb at time i. How I did was something like this:
#let's assume batch_size = 1
initial_dec_input = zeros(1, 1, emb_dim) #All 0s
tgt_emb = zeros(1, tgt_size, emb_dim)
 tgt_emb[0,0, :] = initial_dec_input
for i in range(tgt_size):
  out = model(inp_emb, tgt_emb)
  tgt_emb[0, i+1, :] = out[0,i,:]

Cheers.