Thanks for the update @tom  So essentially, to make sure that the DWT is fixed =, I would simply set dec_hi.requires_grad = False (and same for the others off course). And then everything should work.
Perhaps I’ll write this as a pytorch module so it can be easily integrated into a network.
Thanks again!