So I see a couple of issues, the first being that on the lines where you say “Bottleneck.out_saved[self.counter]” that’s assigning the detached out value to a list which isn’t pegged to this layer, but as far as I can tell is pegged to a list which is shared across all Bottleneck instances? I think what you meant to do is use “self.out_saved[self.counter]”?
I also think there might be an issue with your flow control. You assign to out_saved while not testing but then you increment the counter, such that when you arrive at “elif not test” and try to pull detach from that out_saved[self.counter], I’m not sure that you’re grabbing the correct part of the list. It’s hard to tell from that snippet–I also assume you’re manually assigning “layer.test=True” somewhere in the training code, as opposed to using net.eval()?
BTW the easiest way to quickly debug this yourself would be to just insert a print statement before the line that throws the error and print the type of the output and its value. If your error says “not a Variable,” then what is the output? If it’s a list or a normal python float that could point to the issues I mention above.