Hi,
So in my case I have pretrained a model and after pre-training, in the main training loop, I have applied a few functions (tensor operations) on top of the pre-trained model’s output. This, applying a new function has some parameters that I need to update at each iteration.
Do you recommend making a nn.Module sub-class out of these new functions and parameter where I can define the new parameter using nn.Parameter ??
The problem is, I will not be using any pytorch’s nn layer in this new model (model on top of pre-trained model). So, is it even necessary or the best practice to add additional parameters like this? What do you guys suggest? Please let me know If I am being unclear.
Would appreciate your help, Thanks!