Solved by ptrblck in post #2 
              
              
                I don’t think you are seeing a memory leak, but are allocating additional memory. 
E.g. in your train_one_epoch you are using a pure FP32 forward pass and another one wrapped in autocast: 
        # Iterate over train batches
        for step, data in enumerate(train_bar, start=1):

            # un…