Artificial Intelligence, Machine Learning, Parallel Computing, Diffusion Models, Content Generation.
stores consecutive sets of parameters in operational order in a continuous memory space called Chunk, all the same size, for efficient use of network bandwidth between PCI-e and GPU-GPU, reduced communication, and avoidance of potential memory fragmentation. Colossal-AI’s heterogeneous memory