Thanks! So I this seems to be a general misunderstanding on mutable data types in python (e.g. as explained on StackOverflow 5).
What I still don’t understand is, why the following does not change the state of the cache:
cache = self.state_dict()
weights  = [value for (key,value) in cache.items() if '.weight' in key]
weight_probs = [torch.full_like(w, 0.5) for w in self.layers.parameters()]

with torch.no_grad():
    current = [torch.bernoulli(p) for p in weight_probs]
    for i, layer in enumerate(self.layers.parameters()):
        layer.data = weights[i] * current[i].float()

self.load_state_dict(cache)  # this works, cache seems to be unaltered, why?