I want to implement the following algorithm, from this book, section 13.6 1

Untitled.png954×588 58.5 KB

Therefore, I need to be able to update all the network’s parameters by a constant factor.
I found the following code to acomplish that
state_dict = net.state_dict()

for name, param in state_dict.items():
    # Transform the parameter as required.
    transformed_param = param * 0.9

    # Update the parameter.
    state_dict[name].copy_(transformed_param)

but it doesn’t seem right, that all parameters have to pass through my code, rather than have some vector operation (which is implemented in a low level language, and is cuda supported).
I’m sure there is a better way of doing this.