Hello,
I am training a neural network that outputs a vector: output → with size (Batch size x N).
Then, I want to reduce the vector to (Batch size x M) (len(M)<len(N)) by creating groups of size 10 of N and summing them.
My question is if I do:
new_output = torch.tensor([tensor.sum() for tensor in torch.split(output, 10, dim=1)])
I then have to compare new_output with another variable of size (Batch_size x M) which also required grad.
Will creating a new tensor break Autograd?
Also, how one can be sure that Autograd works correctly? For simple architectures is straightforward, however, I am not sure how to do it if I create more complex ones.
Thanks!