Hey again,
I’m currently developing a transversal machine learning tool that is able to support multiple ML frameworks and therefore I’m doing things a little differently when compared to the regular pytorch workflow.
My model inherits from nn.Module and has the regular init and forward methods. However, to fit the framework, I had to add an update method that calls the forward, computes the loss, calls loss.backward() and calls optimizer.step.
This is the current update bit from my code:
def update(self, input=None, output=None):

    # input_source, input_target, tags, input_editor=None, input_client=None

    input_source = input['source']
    input_target = input['target']
    input_editor = input['editor']
    input_client = input['client']
    tags = output['tags']

    inputs = [input_source, input_target]
    if input_editor is not None:
        inputs += [input_editor]
    if input_client is not None:
        inputs += [input_client]

    if self.cuda:
        inputs = [Variable(inpt.cuda()) for inpt in inputs]
        tags = Variable(tags.cuda())
    else:
        inputs = [Variable(inpt) for inpt in inputs]
        tags = Variable(tags)

    self.optimizer.zero_grad()
    out = self.forward(*inputs)
    loss = self.loss_fn(out, tags)
    loss.backward()
    self.optimizer.step()

    return loss

Everything is working fine, EXCEPT the update bit of the weights. The update method is being called in a train_loop function that calls model.update(**data). I’m printing the losses from every epoch and the values are exactly the same (with different types of data).
Anyone has a clue of what might be wrong?