Well, OK. First let me say that I appreciate your help with this. Second - I got it working. What the update did was kill my CUDA install. Looking at the code I realized that I had “if use_cuda” but not an else statement because I was lazy.
But, your comment about individual parameters has me wondering if I am doing this the best way. So here is the current working code.
use_cuda=torch.cuda.is_available()
device=torch.device(“cuda:0” if use_cuda else “cpu”)
x=X_train
print(“Shape of x”,x.size())
y=Y_labels
print(“Shape of y”,y.size())
print()
loss_array=torch.zeros(epochs,1)
tick=datetime.now()
loss_fn=torch.nn.MSELoss(reduction=‘sum’)
optimizer=torch.optim.Adadelta(net_model.parameters(),lr=1.0,rho=0.9,eps=1e-06,weight_decay=0)
for epoch in range(epochs):
    for i, data in enumerate(train_loader,1):
        inputs, targets = data
        #print(inputs.size(), targets.size())
        #print(epoch, i, data[1])
        
        if use_cuda:             #Use this for all other optimizers
            #print("CUDAing")
            y_pred=net_model(inputs)
            loss=loss_fn(y_pred, targets)
            time=str(datetime.now())  
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            #print('Here')
        else:
            #print("NOT CUDAing")
            y_pred=net_model(inputs)
            loss=loss_fn(y_pred, targets)
            time=str(datetime.now())  
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            #print('Here')                
            
        with torch.no_grad():
            #counter=0
            for param in net_model.parameters():
                param-=alpha*param.grad

    loss_array[epoch]=loss.item()    #Un-comment these 6 lines if NOT using LBFGS
    if epoch%100==0:
        print(datetime.now(),"Loss for epoch:", epoch, loss.item())
    if loss.item() < 1e-8:
        print(datetime.now(),"Loss for epoch:", epoch, loss.item())
        break

Thanks again.