I can fix the problem by replacing the initialization with the following definition.
 def __init__(self):
        super(Test, self).__init__()
        self.model = torch.nn.Sequential(torch.nn.Linear(5, 2)).to(dev)
        self.sms = torch.nn.ModuleList([torch.nn.Linear(2, 2) for _ in range(2)])
# redacted
# def to(self): ... 

Presumably, my earlier self.to is somehow detaching the parameter and interestingly revelas itself only on XLA. I would still very much like to understand what was I doing wrong earlier.