Artificial Intelligence, Machine Learning, Parallel Computing, Diffusion Models, Content Generation.
efficient checkpoint within a given memory budget, rather than just aiming for maximum memory compression. To avoid a lengthy search process for an optimal activation checkpoint, Colossal-AI has implemented a two-stage search process. This allows the system to find a feasible distributed training