That will help partially, but pytorch will still create the computation graph for pre_trained. To avoid this you should detach the output of pre_trained. Like this, for example.
pre_out = pre_trained(input).detach()
out = in_training(pre_out)