Simple API would be a helper function that takes an nn.Module and returns a version that implements the trick above. IE, model = batch_expand(model, N)
Autograd memory usage in new model has to be reasonable, otherwise there’s no benefit over original model and batch-size=1.
This seems to be the case in the example above (colab measurement forum - replication memory - simplified 21)