Hello pvardanis!



 pvardanis:

   ( (x_i**2 + y_i**2) - 1 )**2,

term refers only to the prediction?


Yes.  The “real” loss – the part that also depends on the target – only
cares about the angle of (x, y) – its direction in the x-y plane.  This
term is just to regularize (x, y) by penalizing it if it drifts off the unit
circle.  (There will be no term in the gradient pushing it off the unit circle).


EDIT: Supposing I have a vector of 1024 outputs with (x_i, y_i) pairs I think what you describe should look like this:
    loss_1 = (np.add.reduceat(output ** 2, np.arange(0, len(output), 2)) - 1) ** 2 # penalize if output is outside the unit circle

    loss_1 = torch.from_numpy(loss_1)



This part won’t work as-is.  When you do part of your computation with
numpy, you no longer get the gradients (autograd) “for free.”  If you go
the numpy route, you will have to write your own explicit backward()
function that will let autograd backpropagate the gradients through this
part of your loss.
I don’t know off-hand how to do it, but the cleanest approach would be
to write this entirely with pytorch tensor operations (somehow slicing,
indexing, and/or reshaping to get the (x, y) pairs).  If you do this just
with pytorch tensor functions you will get autograd for free, and you
won’t have to write a backward() function (and it will probably run
faster).


    loss_2 = 1. - torch.cos(output - target) # wrap loss around 2pi



This isn’t right.  First, target is of shape (nBatch, 512), while output
is of shape (nBatch, 1024).  Second, target is a set of angles, while
output is a set of these (x, y) pairs that encode angles, but aren’t
actually angles themselves.
You need to convert your (x, y) pairs to angles with
theta = atan2 (x, y) and package them as a batch, thetas,
of shape (nBatch, 512) with which you will then calculate your
   loss_2 = 1. - torch.cos (thetas - target) 

Again, you should try to do this just with pytorch tensor operations so
that autograd can give you the gradients without your writing an explicit
backward() function.
I imagine that people have worked this all out somewhere already, but
I couldn’t find anything off-hand, and I’m not aware of it being built into
pytorch.
Good luck.
K. Frank