Hello! I have 10000 tuples of numbers (x1,x2,y) generated from the equation: y = np.cos(0.583*x1)+np.exp(0.112*x2). I want to use a NN like approach in pytorch to find the 2 parameters i.e. 0.583 and 0.112 using SGD. Here is my code:
class NN_test(nn.Module):
    def __init__(self):
        super().__init__()
        self.a = torch.nn.Parameter(torch.tensor(0.7))
        self.b = torch.nn.Parameter(torch.tensor(0.02))

    def forward(self, x):
        y = torch.cos(self.a*x[:,0])+torch.exp(self.b*x[:,1])
        return y

model = NN_test().cuda()

lrs = 1e-4
optimizer = optim.SGD(model.parameters(), lr = lrs)
loss = nn.MSELoss()

epochs = 30
for epoch in range(epochs):
    model.train()
    for i, dtt in enumerate(my_dataloader):
        optimizer.zero_grad()
        
        inp = dtt[0].float().cuda()
        output = dtt[1].float().cuda()

        ls = loss(model(inp),output)

        ls.backward()
        optimizer.step()
    if epoch%1==0:
        print("Epoch: " + str(epoch), "Loss Training: " + str(ls.data.cpu().numpy()))

where x contains the 2 numbers x1 and x2. In theory it should work easily, but the loss doesn’t go down. What am I doing wrong? Thank you!