What’s the full error you’re seeing? It doesn’t sound as if it has to do anything with in-place ops. The slowdown comes from the fact, that you’re modifying the W and reusing it in every subsequent iteration, but it keeps it’s history, so it keeps growing. In-place operations are part of history too! If you don’t want to backprop through old graph multiple times, I’d recommend you to repackage W in a fresh Variable after every iteration (see e.g. repackage_hidden 16 from the language modeling example). There should be a better way to solve this, but I’d need to know more about your particular use case.
This also causes the serialization failure - you can only save leaf Variables, but your W keeps hold of the history, so it’s not a leaf!

That’s probably because PyTorch has to pick a more memory-intensive backend for dilated convolutions. If you can send me a diff that I could test, I could confirm that it’s because of this.