Use of AIC’s num. of parameters term is questionable with overparametrized DL models. But likelihood part is comparable across models, yes.



 AlperEkmekci:

I think torch.set_deterministic() and torch.manuel_seed(0) are pretty useful for comparing how changes are affecting to the models. Is it a good way for DL model testing pipeline ?


Manual seed is good, as it has no computational side effects. Additional measures should be used with care, you could slow down stuff without achieving bit-wise reproducibility. IMO, it is more appropriate for continuous hyperparam optimization, to not confuse some search algorithms, discrete model/parameter choices can be more obvious.