The tutorial is ridiculously stupid. Ignore it. Read this:

  
      stats.stackexchange.com
  
  
      
    
  

  Why do attention models need to choose a maximum sentence length? 14



  neural-networks, natural-language, rnn, attention



  asked by
  
  
    Pinocchio
  
  on 01:02AM - 06 Jun 19 UTC


  
  
    
    
  
  


but the point is that their attention layer is fixed matrix. From 2h to max_len as you noted. Note that the attention weights are computed as follow:
exp( target_embedding[t] * self.att ) # (1, self.max_length)

so therefore we don’t have the attention be dependent on the actual representation of the source (since it is NOT using the embeddings of the ecodning source vectors). It’s just an attention based on position, which is stupid and makes no sense. It took no effort to use the encoders as input. Say something like:
exp( target_embeddings[t] * enconder_outputs )
or
attn_weights = F.softmax( embedded[0]*encoder_outputs )

something like that.