That’s not easily doable, since you would need to infer the output shapes of the intermediate activations for all operations. In case you are able to do so, you would also have to stick to all native kernels, since e.g. cudnn uses different workspace sizes for different algorithms, which might be selected if cudnn.benchmark is active.