Hello Granth -



 granth_jain:

Are both torch.tanh and NN.tanh same?


I haven’t checked the code, but I am quite certain that what Alban
said is correct:  Not only do:
torch.tanh (some_tensor)
torch.nn.functional.tanh (some_tensor)
torch.nn.Tanh() (some_tensor)

all perform the same computation (and return the same result), but
they all ultimately resolve to the same implementation to do so.


Which is better to be used as an activation function


The first two versions in my above code are functions.  I believe that
the second is being deprecated in favor of the first (for reasons that
I don’t understand).
The third version is a class whose instances are function objects.
(Note, there is no torch.nn.tanh.  There is a torch.nn.Tanh (a
class) and a torch.nn.functional.tanh (a function) (and the
function torch.tanh).)
If you like using functions, use torch.tanh (some_tensor),
although you can instantiate the function-object version on the fly
and then call it: torch.nn.Tanh() (some_tensor).  (The on-the-fly
function-object approach has the de minimus inefficiency of
instantiating a new object every time the Tanh()() call is made.)
If you want or need to have an instance of a function object, for
example in order to include it as a “layer” in a Sequential, then
you will need to use torch.nn.Tanh.
But again, in terms of the function that gets computed (rather than
the packaging), they are all the same.
Best.
K. Frank