@michaelklachko That was my very first network in PyTorch so keep in mind that I was testing and wasn’t really knowing what I was doing ;). Hence why my dropouts are inconsistent, one in the forward and one in the init.