Solved by ptrblck in post #2 
              
              
                It depends on you and your use case, which part of the model will be fine tuned. 
Andrej Karpathy explained in Stanford’s CS231 you can use the size and similarity of your data as a baseline: 



New dataset is small and similar to original dataset . Since the data is small, it is not a good idea to…