Do you mean passing “fc3_out” directly to loss_fn?
Also, the code below is how I am performing forward and backward propagation. I wonder if this part of the code is causing the gradient not updating:
    y_pred = model(time, freq, clean_result=False)

    # convert to 1 & -1

    for i in range(len(y_pred)):

        if y_pred[i] >= 0.5:

            y_pred[i] = 1

        else: 

            y_pred[i] = 0

    # Compute loss

    loss = loss_fn(y_pred, y)

    # Zero gradients, perform a backward pass, and update the weights.

    optimizer.zero_grad()

    loss.backward()

    optimizer.step()