Yeah, I quoted a number for ancient cards (or old sm, not sure), 200mb is a more reasonable baseline measured above. The point was that pytorch’s context is bigger than that baseline, before kernels are needed.