Hi, I’ve been trying to get Tensorboard working for some recurrent models. I want to visualise the graph for a model, but I have this issue where when I try and use .add_graph, TensorBoard returns:
‘RuntimeError: Cannot insert a Tensor that requires grad as a constant. Consider making it a parameter or input, or detaching the gradient’
As far as I can tell, its complaining about the hidden state of the Recurrent unit. I can make the error go away if I tell the hidden state to detach during ‘forward()’, but then the network is unable to train. Besides, the network is able to run and update parameters properly, so I don’t really see why Tensorboard should be unable to create a graph from it. Thanks! Code is below.
import torch
from torch.utils.tensorboard import SummaryWriter
import torch.nn as nn

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()

        self.rec1 = nn.RNN(1, 10, 1, bias=False)
        self.lin1 = nn.Linear(10,1)
        self.hidden = None

    def forward(self, x):

        x, self.hidden = self.rec1(x, self.hidden)
        #self.hidden = self.hidden.detach()
        return self.lin1(x)


if __name__ == '__main__':

    in_data = torch.empty([100, 10, 1])
    tgt_data = torch.empty([100, 10, 1])

    network = Net()
    loss_fcn = nn.MSELoss()
    optimizer = torch.optim.Adam(network.parameters(), 0.0005)

    network.hidden = torch.empty([1, 10, 10])
    out = network(in_data)

    loss = loss_fcn(out, tgt_data)
    loss.backward()
    optimizer.step()

    writer = SummaryWriter('runs/test_icicles')
    writer.add_graph(network, in_data)