Hi skyline-san,
Do you want to build your custom loss function? PyTorch has standard loss functions, google is your friend.
Tensor is used for variable which is on a path of your neural network (means that non-linear function between linear algebra mappings), implies that should be derivative in case of parameter. Variable is generally used for variable which is not tensor such as input vector.
If you do so, and the “features” means a set of parameter, then you can use nn.Parameter or nn.ParameterList for your custom layer.
If you do not aim so, then you can use not only loss function but also standard layer architectures such as convolution, fully-connected layers, etc, these have its own parameters.
You do not need to concern the gradient matter (“baseline” optimizer) because PyTorch supports auto gradient ability (back-prop is automatically constructed for your network).
I hope this comment help you.
Best,
S.Takano