Yes, you understood correct. CloningLinear is the thing that I was looking for. My buffer is collecting outputs from different episodes that depends on different weight values as they are from different episodes. This approach works for me. Thank you!