ISC - Densifying Assumed-Sparse Tensors - Improving Memory Efficiency and MPI Collective Performance During Tensor Accumulation for Parallelized Training of Neural Machine Translation Models.
arxiv.org