Hi,
I implemented a model which fails to learn. Loss calculated for every epoch is exactly the same and so is the sequence of losses over batches for every epoch. I have been fumbling around with this for a couple of days and apparently stumbled over the reason for this behaviour now.
As it seems all Tensors I use in my C++ code are from namespace at even though I am explicitly creating them with torch::Tensor … . Can anyone please explain why this is so and how to fix this?
I’m including <torch/torch.h> and no other PyTorch headers an I’m never creating any at::Tensors.
I only noticed this because of a compilation error I introduced where the compiler among other things complained about some at::Tensors which I didn’t realize I was using until then.
Any help would be greatly appreciated.
Edit 1:
Calling requires_grad() on any Tensor in my code so far yields false. Guess it should not be that way…
Edit 2: Here’s some code from the training part, maybe it’s just obvious what I’m doing wrong…
torch::optim::SGD optimizer(classifier->parameters(), 0.001);
const unsigned int epochs = 10;
torch::Tensor loss, output, runningLoss;

for(unsigned int epoch=0; epoch<epochs; epoch++) {
	runningLoss = torch::zeros(1);
			
	for(vector<torch::data::Example<>>& batch : *dataLoader) {
		for(torch::data::Example<torch::Tensor, torch::Tensor>& sample : batch) {
			optimizer.zero_grad();
			output = classifier->forward(sample.data);
			loss = torch::mse_loss(output, sample.target);
			runningLoss += loss;
			loss.backward();					
			optimizer.step();
		}
	}			
}

Edit 3:
Maybe I’m just doing something wrong when passing the data to the functions…