Vishu_Gupta:

there seems to be output with type tuple at variable intervals


This would mean that different layers return a variable number of outputs (sometimes a single output would be returned otherwise multiple outputs).



 Vishu_Gupta:

I am not sure if it is ok to ignore them for now but does it make sense to unwrap wherever you have output as type tuple and then detach it like


You should check, if output is indeed a tuple before unwrapping it as otherwise you might slice a tensor or would run into a ValueError. Also, in your current code you are overriding the activation[name] value, so you should consider using different names to store the tuple tensors in the activation dict.