Could you explain this (interesting) use case a bit more?
Why would weight values drift down, if your memory is unreliable?
Wouldn’t this introduce random errors in the memory and thus corrupting the values in an unpredictable way?