Stihl:

out[4] = hid_4[0][-1]


I guess each of these calls are actually modifying out inplace. But it’s value is needed by the dense(out) call?
Can you try not to change out inplace and just torch.cat() your hidden layers together?