I implemented this from scratch based on your description and provided partial code samples and it works for me: the loss value decreases steadily over time. Hope it helps:

  
      gist.github.com
  
  
    https://gist.github.com/crowsonkb/fc08028bc4907820626bea4f34bbe63b 81
ptf.py
"""https://discuss.pytorch.org/t/applying-gradient-descent-to-a-function-using-pytorch/64912"""

import torch
from torch import nn, optim
from torch.utils import data


class NNTest(nn.Module):
    def __init__(self):
        super().__init__()
This file has been truncated. show original