Well here’s a pretty simple problem, how do you go from a
i) classification problem with a single output from your model, and a
loss = nn.CrossEntropyLoss(output, labels)
ii) to a regression problem with a mu, and sigma2 (mean & variance) output from your model, which then goes through
y_pred = torch.normal( mu, sigma2.sqrt() )
and
loss = F.smooth_l1_loss(y_pred, labels)
Basically I want to change a MNIST classifier into regression exercise which outputs a Gaussian distribution. The bit that’s tripping me up is that the output y_pred is now is now stochastic, so I guess I need a .reinforce() on it, but I still don’t not get how to do this?
Here’s the relevant bit of my code,
    def forward(self, x):
        # Set initial states
        h0 = Variable(torch.zeros(self.num_layers*2, x.size(0), self.hidden_size)) # 2 for bidirection 
        c0 = Variable(torch.zeros(self.num_layers*2, x.size(0), self.hidden_size))
        
        # Forward propagate RNN
        out, _ = self.lstm(x, (h0, c0))
        
        # Decode hidden state of last time step
        mu = self.mu( out[:, -1, :] )
        sigma2 = self.sigma2( out[:, -1, :] )
        return mu, sigma2

rnn = BiRNN(input_size, hidden_size, num_layers, num_classes)

# Loss and Optimizer
optimizer = torch.optim.Adam(rnn.parameters(), lr=learning_rate)
    
# Train the Model 
for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(train_loader):
        images = Variable(images.view(-1, sequence_length, input_size))
        labels = Variable( labels.float() )        
        # Forward + Backward + Optimize
        optimizer.zero_grad()
        #outputs = rnn(images)
        mu, sigma2 = rnn(images)
        sigma2 = (1 + sigma2.exp()).log() # ensure positivity
        y_pred = torch.normal( mu, sigma2.sqrt() ) 
        y_pred = y_pred.float()
        #y_pred = Variable( torch.normal(mu, sigma2.sqrt()).data.float() )       
        loss = F.smooth_l1_loss( y_pred , labels )            
        loss.backward()
        optimizer.step()


and the compile error,
  File "main_v1.py", line 90, in <module>
    loss.backward()
  File "/home/ajay/anaconda3/envs/pyphi/lib/python3.6/site-packages/torch/autograd/variable.py", line 158, in backward
    self._execution_engine.run_backward((self,), (gradient,), retain_variables)
  File "/home/ajay/anaconda3/envs/pyphi/lib/python3.6/site-packages/torch/autograd/stochastic_function.py", line 13, in _do_backward
    raise RuntimeError("differentiating stochastic functions requires "
RuntimeError: differentiating stochastic functions requires providing a reward

It’s modified from
yunjey/pytorch-tutorial/blob/master/tutorials/07%20-%20Bidirectional%20Recurrent%20Neural%20Network/main.py
OR, perhaps I’m making it more complicated than it needs to be with the Gaussian thing? Should I just stick an encoder on the output of the LSTM ???
Thanks a lot