snip3r77:

So basically , we can re-use the weights but we need to change the architecture based on this function


You shouldn’t change the architecture, but recreate the model using the same arguments as during the initial training.



 snip3r77:

What’s the difference if we re-train again when we define the network? In what way we’re re-using
checkpoint['state_dict'] and how it differs with transfer learning?


Loading the state_dict is used to load all previously stored (and trained) parameters and buffers into the model. It’s the first step when you want to fine tune the model (transfer learning).



 snip3r77:

I’ve been going from coursera deeplearning.ai , fast.ai and now udacity’s pytorch. It seems like I prefer pytorch ( or fast.ai - abstract short codes ) and I love the community here.
My question is how do I get used to the syntax
( i.e turn off gradient and dropout during inferencing ).
Thank you for the reply.
What I’m trying to say is there is a lot of steps that are consistent I think, so how does one remember these steps OR usually we do not need to remember ( just keep it in your notepad) and understand what it is trying to do. Because DL is definitely not as straight forward as sklearn’s .fit and .predict.


That’s great to hear!
Once you’ve created a training, validation and testing loop, you might realize that there are not that many things to remember.
Some common issues are, if you forget to call optimizer.zero_grad() (which still happens to me after some time using PyTorch  ), or to forget to call model.train()/.eval().
That being said, I’m not sure if there are any gotchas, but let us know, if you think I’m missing something.