Solved by Vladimir_Iashin in post #2 
              
              
                Scaler (optimizer) looks for parameters used in the graph which is empty, hence, the error. 
This specific example was solved by replacing 
x.requires_grad = True 
with 
x = x + 0 * self.__hidden__(x)