Training code is as follows;
model.train()
  
for epoch in range(EPOCH):
  for x, t in dataloader_train:
    t = t.to(device)
    for time in range(TIME_STEPS):

      x_ = x[0][0][time]
      x_ = x_.to(device)
      for index_a in range(NUM_INPUT):
        for index_b in range(NUM_HIDDEN-1, 0, -1):
          model.fw_x[index_a][index_b] = model.fw_x[index_a][index_b - 1]
          model.fw_h[index_a][index_b] = model.fw_h[index_a][index_b - 1]
      
        model.fw_x[index_a][0:NUM_INPUT] = x_
        model.fw_h[index_a][0] = 0.0

      model.zero_grad()
      y = model()
      loss = criterion(y, t)
      loss.backward(retain_graph = True)
      optimizer.step()

Is such the FIFO coding not allowed? Then how to write same function with PyTorch
grammer?