Regarding the loss functions, you can even use a different one at each epoch without problem: if I’m not mistaken the only condition is that they are differentiable (=>and do not break the computation graph)