Hello Haroune and Henry!



 mohammedi-haroune:

how can I differentiate between classes that constitute an NN model and those which implement losses


I don’t think that there is a simple, one-stop-shopping way of doing
this that is completely reliable.  Note that activations also inherit
from Module.
Losses, however, (appear to) inherit (possibly indirectly)  from
torch.nn.modules.loss._Loss (which, in turn, inherits from
Module).  You could look for that.
You might consider counting parameters() on the theory that any
self-respecting model has parameters (although some activation
functions also have parameters).
Also, you have to be clear about what you mean by a “model.”
Would you consider a single Linear to be a model?  Arguably
it would be.  How about a single Softmax().  This seems more
of a stretch, but how does it behave any differently than a
Linear?  (One difference is that it doesn’t have any parameters.)
This script illustrates some of these points:
import torch
print (torch.__version__)
model_linear = torch.nn.Linear (3, 5)
model_sequential = torch.nn.Sequential ((torch.nn.Linear (4, 6)))
loss_mse = torch.nn.MSELoss()
loss_ce = torch.nn.CrossEntropyLoss()
act_softmax = torch.nn.Softmax()
act_prelu = torch.nn.PReLU()
print (sum (1 for _ in model_linear.parameters()))       # count parameters
print (sum (1 for _ in model_sequential.parameters()))   # count parameters
print (sum (1 for _ in loss_mse.parameters()))           # count parameters
print (sum (1 for _ in loss_ce.parameters()))            # count parameters
print (sum (1 for _ in act_softmax.parameters()))        # count parameters
print (sum (1 for _ in act_prelu.parameters()))          # count parameters
print (model_linear.__class__.__bases__)                 # immediate superclass
print (model_sequential.__class__.__bases__)             # immediate superclass
print (loss_mse.__class__.__bases__)                     # immediate superclass
print (loss_ce.__class__.__bases__)                      # immediate superclass
print (act_softmax.__class__.__bases__)                  # immediate superclass
print (act_prelu.__class__.__bases__)                    # immediate superclass
print (model_linear.__class__.__mro__)                   # full class hierarchy
print (model_sequential.__class__.__mro__)               # full class hierarchy
print (loss_mse.__class__.__mro__)                       # full class hierarchy
print (loss_ce.__class__.__mro__)                        # full class hierarchy
print (act_softmax.__class__.__mro__)                    # full class hierarchy
print (act_prelu.__class__.__mro__)                      # full class hierarchy

Here is the output:
>>> import torch
>>> print (torch.__version__)
1.6.0
>>> model_linear = torch.nn.Linear (3, 5)
>>> model_sequential = torch.nn.Sequential ((torch.nn.Linear (4, 6)))
>>> loss_mse = torch.nn.MSELoss()
>>> loss_ce = torch.nn.CrossEntropyLoss()
>>> act_softmax = torch.nn.Softmax()
>>> act_prelu = torch.nn.PReLU()
>>> print (sum (1 for _ in model_linear.parameters()))       # count parameters
2
>>> print (sum (1 for _ in model_sequential.parameters()))   # count parameters
2
>>> print (sum (1 for _ in loss_mse.parameters()))           # count parameters
0
>>> print (sum (1 for _ in loss_ce.parameters()))            # count parameters
0
>>> print (sum (1 for _ in act_softmax.parameters()))        # count parameters
0
>>> print (sum (1 for _ in act_prelu.parameters()))          # count parameters
1
>>> print (model_linear.__class__.__bases__)                 # immediate superclass
(<class 'torch.nn.modules.module.Module'>,)
>>> print (model_sequential.__class__.__bases__)             # immediate superclass
(<class 'torch.nn.modules.module.Module'>,)
>>> print (loss_mse.__class__.__bases__)                     # immediate superclass
(<class 'torch.nn.modules.loss._Loss'>,)
>>> print (loss_ce.__class__.__bases__)                      # immediate superclass
(<class 'torch.nn.modules.loss._WeightedLoss'>,)
>>> print (act_softmax.__class__.__bases__)                  # immediate superclass
(<class 'torch.nn.modules.module.Module'>,)
>>> print (act_prelu.__class__.__bases__)                    # immediate superclass
(<class 'torch.nn.modules.module.Module'>,)
>>> print (model_linear.__class__.__mro__)                   # full class hierarchy
(<class 'torch.nn.modules.linear.Linear'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (model_sequential.__class__.__mro__)               # full class hierarchy
(<class 'torch.nn.modules.container.Sequential'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (loss_mse.__class__.__mro__)                       # full class hierarchy
(<class 'torch.nn.modules.loss.MSELoss'>, <class 'torch.nn.modules.loss._Loss'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (loss_ce.__class__.__mro__)                        # full class hierarchy
(<class 'torch.nn.modules.loss.CrossEntropyLoss'>, <class 'torch.nn.modules.loss._WeightedLoss'>, <class 'torch.nn.modules.loss._Loss'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (act_softmax.__class__.__mro__)                    # full class hierarchy
(<class 'torch.nn.modules.activation.Softmax'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)
>>> print (act_prelu.__class__.__mro__)                      # full class hierarchy
(<class 'torch.nn.modules.activation.PReLU'>, <class 'torch.nn.modules.module.Module'>, <class 'object'>)

Best.
K. Frank