I am using the Rprop optimizer and MSE for my loss function but am finding zero_grad isn’t setting the values for grad to zero for the network and the MSE.backward() doesn’t update the values either. I must be doing something very silly.
model = Net(feature_number=2, hidden_number=6, output_number=1)
search_method = torch.optim.Rprop(model.parameters(), lr=0.02)
## ommitted
def local_search(weights, network, loss_function, search_method , iterations, x_train, y_train, approach=LEARNING_METHOD):
    """ Perform local search for a given number of iterations """
    weights_into_network(network, weights)
    for i in range(iterations):
        output = network(x_train)
        loss = loss_function(output, y_train)
        search_method .zero_grad()
        loss.backward()
        search_method .step() #<--- here thrown
        learned_loss = loss.item() 
## ommitted

  File "C:\Users\user\nn_ga\main.py", line 381, in local_search
    search_method.step()
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\optim\optimizer.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\autograd\grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\optim\rprop.py", line 109, in step
    step_size_min=step_size_min,
UnboundLocalError: local variable 'step_size_min' referenced before assignment

Versions of relevant libraries:
[pip3] numpy==1.21.1
[pip3] torch==1.10.1+cu102
[pip3] torchaudio==0.10.1+cu102
[pip3] torchvision==0.11.2+cu102


  

      github.com/pytorch/pytorch
  

  
    
  
	  
  

  
    
      Rprop Optimizer: UnboundLocalError: local variable 'step_size_min' referenced before assignment 1
    

    
      
        opened 
        
          
        
        Dec 23, 2021
      
      


      
        
          
          priceluke
        
      
    

    
        
          module: optimizer
        
        
          triaged
        
    
  


  
    ### 🐛 Describe the bug

When using the optimiser's step method, I get an error r…egarding step minimums. I believe this value should default to 1e-6 if not explicitly set via params.

```
model = Net(feature_number=2, hidden_number=6, output_number=1)
search_method = torch.optim.Rprop(model.parameters(), lr=0.02)
## ommitted
def local_search(weights, network, loss_function, search_method , iterations, x_train, y_train, approach=LEARNING_METHOD):
    """ Perform local search for a given number of iterations """
    weights_into_network(network, weights)
    for i in range(iterations):
        output = network(x_train)
        loss = loss_function(output, y_train)
        search_method .zero_grad()
        loss.backward()
        search_method .step() #<--- here thrown
        learned_loss = loss.item() 
## ommitted
```

```
  File "C:\Users\user\nn_ga\main.py", line 381, in local_search
    search_method.step()
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\optim\optimizer.py", line 88, in wrapper
    return func(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\autograd\grad_mode.py", line 28, in decorate_context
    return func(*args, **kwargs)
  File "C:\Users\user\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\optim\rprop.py", line 109, in step
    step_size_min=step_size_min,
UnboundLocalError: local variable 'step_size_min' referenced before assignment
```


### Versions

Versions of relevant libraries:
[pip3] numpy==1.21.1
[pip3] torch==1.10.1+cu102
[pip3] torchaudio==0.10.1+cu102
[pip3] torchvision==0.11.2+cu102

cc @vincentqb @jbschlosser @albanD