Maybe you want to register a buffer 30 instead?
Parameters are meant to be optimised. If you don’t want a parameter, simply use a Variable instead, which is roughly equivalent to your grad-less Parameter.
This should do the trick.
import torch

from torch import Tensor
from torch.nn.parameter import Parameter
from torch.autograd import Variable
from torch.nn import Module

class Blah(Module):
    def __init__(self, dim):
        super(Blah, self).__init__()
        self.s = Variable(torch.rand(1, dim))
        self.t = Parameter(torch.rand(1, dim))

blah = Blah(10)

blah.zero_grad()