This was fixed on master via https://github.com/pytorch/pytorch/commit/a76098ac1532d5e9ee24b4776258ae731627f8e3 134
and will be fixed in the next release.
For now your workaround is:
import torch.nn as nn
import torch.optim as optim

if __name__ == '__main__':

    module = nn.Sequential(nn.Linear(2,3),nn.Linear(3,2))
    params_dict = dict(module.named_parameters())
    params = []
    for key, value in params_dict.items():
        if key[-4:] == 'bias':
            params += [{'params':[value],'lr':0.0}]
        else:
            params += [{'params':[value],'lr':0.1}]
    op = optim.SGD(params, momentum=0.9)